BOUNDARY BEHAVIOR OF SPECIAL COHOMOLOGY CLASSES 
ARISING FROM THE WEIL REPRESENTATION 
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Abstract. In our previous paper [11], we established a theta correspondence be- 
tween vector- valued holomorphic Siegel modular forms and cohomology with local 
coefficients for local symmetric spaces X attached to real orthogonal groups. This 
correspondence is realized via theta functions associated to explicitly constructed 
"special" Schwartz forms. Furthermore, the theta functions give rise to generat- 
ing series of Poincare dual classes of certain " special cycles" in X with coefficients 
arising from embeddings of smaller orthogonal groups. 

In this paper, we compute the restriction of these theta functions to the Borel- 
Serre boundary of X. The global restriction is again a theta series as in [11], now 
for a smaller orthogonal group and a larger coefficient system. 

As application we establish the cohomological nonvanishing of the special (co)cycles 
when passing to an appropriate finite cover of X. In particular, the (co)homology 
groups in question do not vanish. 



1. Introduction 

The cohomology of arithmetic quotients of a symmetric space D associated to a 
reductive Lie Group G is of fundamental interest in number theory and for the field of 
automorphic forms. One very attractive topic in this area is the construction of non- 
trivial cohomology classes via the embedding of H G for suitable subgroups H. 
Classes of these geometrically defined cycles are often called (generalized) modular 
symbols or also "special" cycles. 

Since the work of Hirzebruch and Zagier [16] for algebraic cycles in Hilbert modular 
surfaces and Shintani [25] for the "classical" modular symbols inside modular curves, 
the occurrence of intersection numbers of geometrically defined cycles as Fourier co- 
efficients of automorphic forms has been widely studied. Kudla and Millson (see e.g. 
[20]) carried out an extensive program to explain the work of Hirzebruch-Zagier from 
the point of view of Riemannian geometry and the theory of reductive dual pairs and 
the theta correspondence. They obtain analogues of the results of [16] for orthogonal, 
unitary, and symplectic groups of arbitrary dimension and signature. In particular, 
they obtain a lift from the cohomology of compact supports for the associated locally 
locally symmetric spaces to the space of holomorphic Siegel and Hermitian modular 
forms. Another important work in this context is the singular theta lift introduced by 
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Borcherds [3] in the orthogonal case. In [8], a duaUty result between the Borcherds 
and the Kudla-Millson lift is given. 

In our previous paper [11], we extended the Kudla-Millson theta lift and obtained 
a correspondence between vector-valued holomorphic Siegel modular forms and coho- 
mology with local coefficients for local symmetric spaces X attached to real orthogonal 
groups. This correspondence is realized via theta functions associated to explicitly 
constructed "special" Schwartz forms. These forms give rise to relative Lie algebra 
cocycles for the orthogonal group with values in the Weil representation tensored 
with the coefficients. Furthermore, the theta functions give rise to generating series 
of Poincare dual classes of certain "special" cycles in X with coefficients arising from 
embeddings of smaller orthogonal groups. 

In this paper, we systematically study certain complexes associated to the Weil 
representation and introduce a "local" restriction map for the Weil representation to 
a face of the Borel-Serre enlargement of the underlying orthogonal symmetric space. 
In particular, we obtain explicit formula for the "local" restriction of the "special" 
cocycles. We then compute the restriction of these theta functions to the Borel-Serre 
boundary of X. The global restriction is again a theta series as in [11], now for a 
smaller orthogonal group and a larger coefficient system. 

As application we establish the cohomological nonvanishing of the special (co) cycles 
in a certain range when passing to an appropriate finite cover of X. 

Furthermore, we indicate how one can use our work to obtain a different, somewhat 
more topological proof of the results of Hirzebruch-Zagier. This will be carried out 
in a different paper [12]. 

We also mention that [10] contains an introductory survey of the results obtained 
in this paper. 

1.1. Statement of the main result. Let V be a rational non-degenerate quadratic 
space of dimension m and signature (p, q) and let G_ — SO(V^). We let G = G(M)o — 
SOo(T^). Let Dy = D = G/K he the symmetric space of G with K = Ky a maximal 
compact subgroup. We let g = 1^ © p be the associated Cartan decomposition of the 
Lie algebra of G associated to a Cartan involution 6. We let F be an appropriate 
congruence subgroup oiG_{'L)r\G and write X — Xy — V\D for the associated locally 
symmetric space. We let G' — Mp(n, R) denote the metaplectic covering group of the 
symplectic group Sp(n, M), and we let K' be the 2-fold covering of U(n) in G' . Note 
that K' admits a character det^^^^ of K' . We let U = /\"(C")* and define an action 
of K' on the tensor power T^{U) by requiring that K' acts on T^{U) by det~-'~^~^^'^^. 
Furthermore, we let >V„,y be the i^'-finite vectors of the Weil representation of G' x G. 

Every partition A of a non-negative integer into at most n parts gives rise to a 
dominant weight A of GL(n). Wc write i(A) for the number of nonzero entries of A. 
We explicitly realize the corresponding irreducible representation of highest weight A 
as the image §a(C") of the Schur functor Sa(-) for (a standard filling of) the Young 
diagram D{\) associated to A applied to the tensor space T^' [C^]. We can apply the 
same Schur functor to T^' {Vc) to obtain the space §a(^c)- We let U : T^' {Vc) V^'^ 
be the orthogonal projection to the harmonic £ -tensors . Then, if the sum of 
the lengths of the first two columns of D{X) is at most m, applying H. to Sa(Vc) one 
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obtains the nonzero irreducible representation S[a](Vc) for 0(Vc:), see [9] section 19.5. 
Furthermore, this gives rise to an irreducible representation for G of a certain highest 
weight A (unless m is even and i{X) — y, in that case it sphts into two irreducible 
representations). If i{X) < [y], then A has the same nonzero entries as A (when A is 
expressed in coordinates relative to the standard basis {ej} of [7], Planche 11 and IV). 

Note that in our previous work [11], we proceeded the other way starting from a 
dominant weight A for G to obtain a weight A' for GL(n, C) such that A and A' have 
the same nonzero entries (and subsequently did not distinguish between A and A' 
writing A = A'). For the purposes of this paper going from GL(n, C) to G is more 
convenient. The results of [11] are still valid in this more general setting. 

The main point of [11] was the construction of certain (g, ii')-cocycles 

where C^''^' is the complex given by 

(1.2) Cj;'-'!^! = [T^{U) ® §a(C")* ® Wn,v ® A'\D) ® §[A](Vt )]'"'"'' 
~ \t^{U) ® Sa(C")* (8) Wny ® A (Pc) ® S[A](^c)J 

Here A"''^{D) denotes the differential ng-forms on D. The isomorphism is given by 
evaluation at the base point of D. Furthermore, K' acts on the first three tensor 
factors, while G (resp. K) acts on the last three. The differential is the usual relative 
Lie algebra differential for the action of OiV). These forms vanish if n > p. 

These classes generalize the work of Kudla and Millson (see e.g. [20]) to the case of 
nontrivial coefficients systems S[a](Vc). In [11], we showed how theta series ^(v'^gjAj) 
associated to j^^j give rise to holomorphic vector- valued Siegel modular forms whose 
Fourier expansions involve periods over certain "special" cycles with coefficients. 

The purpose of this paper is to study the boundary behavior of these classes. 

We let P = P(M)o be the connected component of the identity of the real points of 
a rational parabolic subgroup P in G stabilizing a flag F of totally isotropic rational 
subspaces in V. We can choose F in such a way such that the ^-stable subgroup of 

P forms a Levi subgroup. We let P = NAM be the associated (rational) Langlands 
decomposition, and wc let m and n be the Lie algebras of M and N respectively. We 
set pM = p n m. Let E be the largest element in the isotropic flag F with dimension 
i. We let W — E-^/ E, which is naturally a quadratic space of signature {p — l,q — £), 
and we realize W as a, subspace of V such that the Cartan involution 6 for OiV) 
restricts to one for 0(14^). Then M splits naturally into the product of SOo(VrR) 
with a product of special linear groups of subquotients of Pr, and consequently we 
have a projection map from the symmetric space Dm for M to Dw, the symmetric 
space for SOo(WiR). We let e(P) = NM/Kp be the associated face of the Borel-Serre 
enlargement of D with Kp — M O K, see [4]. 
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We consider an analogous complex A'p'^^ at the boundary given by 
(1.3) = [T^{U)^^,{C^r^Wn,w^A'{e{P))^S[,]{Vc)f '''''' 

T\U) ® Sa(C")* ® Wn,w ® A'^*^ ® P^)c ® S[A](^c)] 

with coefficients in the Weil representation for G' x 0(W]r). We introduce a local 
restriction map of complexes 



(1.4) rp:C'y^ A* 



induced by a natural (restriction) map A^(P*) ~^ A^(^ ® Pm)* and by a G' x NM- 
intcrtwiner from Wn,v to Wn,w using a so-called mixed model of Wny- 

We also construct an inclusion map of complexes from the relative Lie algebra 
complex for W into A*p with the property 

Here ujn — (1, . . . , 1) is the n-th fundamental weight for GL(n), so that the Young 
diagram associated to itUn is an n by £ rectangle. The map Lp is a G" x 0{Wk)- 
intcrtwincr. The map ip is induced by the pullback of Dw to Dm and the explicit 
construction of an embedding of §[fro„+A](W^c) into the cocylcs in the nilpotcnt Lie 

algebra complex A (n^) ® >S'[a](Vc). In fact, we have an embedding of M-modules 

(1-6) S[^^„+A](W^c) H-'in, S[,](yc)) 

into the nilpotent cohomology. 

We exphcitly describe the local restriction rp{ip^^^^-^) in the complex A*p. As a 
consequence we obtain 

Theorem 1.1. Let i{X) <n<p. Then the cohomology class ['rp{(f^q ^x])] satisfies 

['rp{'Pnq,[X])] = [^p(V^5g-£),[te„+A])]- 

In particular, [rp{(p^g [;^])] = for n > min (p, [y]) — £ (if £ > 2) and n > p — 1 or 
n >m-2-i{\) (if £ = I). 

On the other hand, let P he a (real) parabolic subgroup as above such that the 
associated space W is positive definite. Assume 



(1.7) i(A) <n< 

Then 



p-l-i(A) ifq = l. 



[^P(<[A])] 7^ 0. 

The connection between the local restriction map and the global restriction to a 
face of the Borel-Serre compactification X is the following. We let e'(P) be the corner 
corresponding to e(P) in the Borel-Serre compactification X of X, and let fp be the 
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restriction map from X to the corner e'(P). There is a global version of (1.5) which 
on the level of cohomology induces a map 

(1.8) Tp : H^^^-'\Xw, §K^„+A](W^c)) H<'i-'\XM, H^\n, §[a](^c))) 

^//"''(e'(P),§[,](Vt)). 
Here Xm and Xw are suitable arithmetic quotients of Dm and Dw respectively. 

We let Ocy{g', <p]^q ^x]) ^^^^ 9' ^ ^' ^® theta series associated to the Schwartz 
form 9?„g,[A] and a theta distribution associated to a suitable (coset of a) lattice jCy 
in V stabilized by F viewed as a closed differential ng-form on X. The theta series is 
termwise moderately increasing. However, switching to the mixed model of the Weil 
representation we show 

Theorem 1.2. The form Ocyi^nq [A]) contends to a differential form on the Borel-Serre 
compactification X . Moreover, for a given face e'{P), there exists a theta distribution 
J^w for W such that 

[MOcA'Pn,,lX]))] = [%^(^p(¥^n.,[A]))] 

and 

Loosely speaking Theorem 1.1 and Theorem 1.2 can be summarized that the re- 
striction of our theta series for 0{V) to a face of the Borel-Serre compactification is 

the theta series for 0{W) of the same type corresponding to an enlarged coefficient 
system corresponding to placing an n by £ rectangle on the left of the Young diagram 
corresponding to A to obtain a bigger Young diagram corresponding to £wn + A. 

1.2. Applications. 

1.2.1. Nonvanishing of the special cycles. Restricting to a face e'(P) such that the 
associated subspace 14^ of y is positive definite, we obtain 

Theorem 1.3. Assume that the Q-rank and the M.-rank of G coincide. Then in the 
nonvanishing range (1.7) of Theorem 1.1, there exists a finite cover X' of X such 
that 

[^(^i,[A])] 7^ 0. 

In particular, 

iJ"^(X',§[,](Vc))^0. 

Furthermore, H"-'^{X', S[a](Vc)) is not spanned by classes arising from invariant forms 
on D. 

When viewed as a function of r G EI„, the Siegel upper half space, it is shown 
in [11] that the theta series [(^Cv{'^-:V^q[\])] is a holomorphic vector- valued Siegel 
modular form for the representation S'a(C'*) (8)det"*'^^ with values in H"''^{X, S[a](Vc)). 
The Fourier expansion of such a form is parametrized by (integral) positive semi- 
definite matrices T. Furthermore, for T positive definite, the T-th Fourier coefficient 
of 9cy{T, ifi^^ j^j) represents a Poincare dual form for certain special cycles 

(1.9) Zt,ix] e Hom (5a(C'^), i^p(n-g)(^, dX, S[x]{Vc))) . 
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These cycles arise from embedded orthogonal groups associated to positive definite 
rational subspaces U C V oi signature (p — n,q). Namely, the embedding of the 
stabilizer Gu of such a subspace in to G defines a subsymmertic space Du in D and 
a cycles Zu = (F fl Gu)\Du in X. For x = {xi, . . . , a;„) G C, we let (x, x) = (xj, Xj)ij 
be its inner product matrix. Then (for trivial coefficients), the cycles are given by 

(1-10) Zt ^ ^ ■^span(x)- 

xer\£ 
i(x,x)=r 

We then obtain 

Corollary 1.4. Under the hypotheses of Theorem 1.3, there exists a finite cover X' 
of X such that 

for infinitely many T in cohomology. 

Remark 1.5. Bergeron [1] in the compact case established nonvanishing of the Kudla- 
Millson classes (with trivial coefficient system) by considering the analogous classes 
in U{p, q) and using the nonvanishing result in the unitary case by Kazhdan, see [6]. 

Li [22] also used the theta correspondence to establish nonvanishing for the coho- 
mology of orthogonal groups, again in the compact (or L^)-case (without giving a 
geometric interpretation of the classes) . 

Speh and Venkataramana [26] gave in general a criterion for the nonvanishing of 
certain modular symbols on locally symmetric spaces in terms of the compact dual. 
In contrast to our result, their nonvanishing occurs from classes defined by invariant 
forms on D. 

For compact hyperbolic manifolds (when q — 1), Millson [23] proved the cohomo- 
logical nonvanishing of the special cycles in codimension n with nontrivial coefficients 
in the range i(A) < n < p — i{X). The shift in the noncompact case for the upper 
bound for nonvanishing is of course expected by Borel-Serre [5]. 

1.2.2. Hirzebruch-Zagier. We first describe the main motivation for our work. 

The theta series 9c{^nq,o) (for simplicity, we only consider here the trivial coefficient 
system) gives rise to a hft 

(1-11) Ang{v,r)= [ V^Oc{^ng,o){r) 

Jx 

for T] a closed (p — n)g-form on X with compact support. This map factors through 
cohomology, and we obtain a map 

(1.12) An, : H^f-"^'^{X.^ C) ^ Mt\T') 

2 

from the cohomology with compact supports to the space of holomorphic Siegel mod- 
ular forms of degree n of weight m/2 for a certain congruence subgroup F' C Sp(n, Z). 

Our work is motivated by the problem to extend the lift by Kudla and Millson 
(and our extension to non-trivial coefficient systems) from the cohomology of compact 
supports to cohomology groups of the space X which capture its boundary. 
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The point is that the original work of Hirzebruch-Zagier provides such a lift from 
the full cohomology of a Hilbert modular surface. For the following, see [16, 27]. 
For signature (2,2) and Q-rank 1, X is a Hilbert modular surface, and the cycles 
in question for n = 1 are the famous Hirzebruch-Zagier curves T^r parameterized by 
positive integers A^. These cycles define classes in H^'^{X, Q). Hirzebruch-Zagier con- 
sider the smooth (toroidal) compactification X oi X. Then the cohomology H'^{X, Q) 
naturally splits into the orthogonal direct sum of the classes obtained by the compact- 
ifying divisors and the image of H^(X, Q) inside H'^{X, Q) induced by the projection 
X ^ Y, the Baily-Borel compactification of X. Hirzebruch-Zagier define the cycle 
by the projection onto the part coming from the interior of X. They then show by 
explicit calculation of intersection numbers that the generating series of the truncated 
cycles is a holomorphic modular form of weight 2. In particular, for fixed M > 0, 

oo 

(1.13) Pm(t) := ^(rM,T^)e'^^^^ G M^iV). 

N=0 

Since the curves Tm generate H^'^{X,Q), one obtains a lift 

(1.14) P : H^'\X,Q) ^ M2iT'). 

Moreover, the holomorphic form Pm can be written as the sum of two non-holomorphic 
modular forms 

(1.15) PM{r)^HM{T) + lM{r), 

where Hm incorporates the intersection numbers of Tjv and Tm in the interior of X 
(suitably defined) and Im the boundary contributions. 

In a different paper [12], we will obtain the work of Hirzebruch-Zagier from a 
topological point of view using our current work. We give a brief sketch. 

The theta series Ocy{V2o) associated to the given data now has weight 2. In this 
case, the restriction of the classes Ocvi^2,Q) to any boundary face e'(P) is a theta 
series for the associated space W of signature (1, 1). We have 

Theorem 1.6. The restriction of Ocvi'^Xo) to dX gives an exact differential form on 
dX. For each boundary face e'{P) there exists a theta series 6^ (0^) for signature 
(1, 1) of weight 2 for a certain e LF'iW^ with values in the differential 1-forms on 
e{P) such that ^£^(0^) is a primitive for rpOcyifXfi) on the boundary face e'{P): 

d{0cjr))=rp9cAvlo)- 

Note this also shows that the hypotheses of Theorem 1.3 are necessary in general. 

The boundary faces are all isolated, and for simplicity we now assume that there is 
only one cusp, that is, dX — e'{P) for one parabolic P. Via the usual mapping cone 
construction we then view the pair [^£y (</?^o)> %^(^^)] ^ element of H^{X,C). 
Explictly, let rjhe a closed 2-form on the Borel-Serrc compactification X representing 
a class [rj] in the full cohomology H'^{X, C). The pairing between [Ocv{'^2,o)^ ^£tt.(^^)] 
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and [f]] is given by 

J X JdX 

This provides the envisioned extension of the geometric theta hft. In particular, 
we recover Hirzebruch-Zagier: 

Theorem 1.7. The pairing ([r^], [Ocv{vXfi)-i ^Cw^^^^^^ takes values in the holomorphic 

modular forms of weight 2. Hence the pair [Ocv{^X,o)i ^Cw^'^^^'^ induces a map on the 
full cohomology of X to the space of holomorphic modular forms: 

A : H\X,C) M2(r'). 
Moreover, when viewed as an element of H^{X ,C), we have 

N>0 

In particular, 

and the two integrals are the two summands Hm and Im of Pm, see (1.15). 

There are other instances when the restriction to the boundary vanishes in coho- 
mology at each face, and it is natural to expect that a similar analysis can be carried 
out. Particularly attractive is the case n = p — 1 for quotients of the Hermitian 
domain associated to 0(p, 2). In this situation the cycles are modular or Shimura 
curves. 

In general, we anticipate that the interpretation of the special classes in weighted 
cohomology should be very fruitful, see Remark 9.6. In this context note that for 
p > q, the classes are square integrable if p > 2n + 1. 

The paper is organized as follows. In section 2, we establish the basic notation of 
the paper. In particular, we introduce the locally symmetric space X and its Borel- 
Serre compactification and give an explicit description of the parabolic subgroups of 
G. In section 3, we briefly review some basics of the representation theory of GL(n, C) 
and 0(Vc) via the Schur functor §(■). In section 4, we study various models of the 
Weil representation and introduce the Weil representation restriction map r^ from 
S{V^) to S{W^) and study rp^ for a certain class of Schwartz functions. In section 
5, we introduce the complexes Cy and A'p and define the map tp from C'y to A'p. In 
section 6, we study certain aspects of the cohomology of A'p arising from the fiber Np. 
In particular we introduce the map Lp (1.5). In sections 7 and 8, we study the special 
Schwartz form ^nq,ix\ (and variants of it) and establish a first version of the local 
restriction formula on the level of differential forms and then establish Theorem 1.1. 
In section 9, we turn to the global situation. We introduce the theta series ^(v5^y ^^l) 
and prove Theorem 1.2. From this, we obtain the nonvanishing result Theorem 1.4. 

We would like to thank G. Gotsbacher, L. Saper, and J. Schwermer for fruitful 
discussions and also E. Freitag and R. Schulze-Pillot for answering a question on 
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positive definite theta scries. As always it is a pleasure to thank S. Kudla for his 
encouragement . 

The work on this paper has greatly benefitted from three visits of the first named 
author at the Max Planck Institute from 2005 to 2008. He gratefully acknowledges 
the excellent research environment in Bonn. 

2. Basic Notations 

2.1. Orthogonal Symmetric Spaces. Let F be a rational vector space of dimen- 
sion m = p + q and let ( , ) be a non-degenerate symmetric bilinear form on V with 
signature {p, q). We fix a standard orthogonal basis ei, . . . , ep, Cp+i, . . . , of 1^ such 
that (cq, Ca) = 1 for 1 < a < p and (e^, e^) = —1 ioi p + 1 < ii < m. (We will use 
"early" Greek letters to denote indices between 1 and p, and "late" ones for indices 
between p-\-l and m) . With respect to this basis the matrix of the bilinear form is 

given by the matrix Ip^q — {^^ 

We let G_ — SO(y) viewed as an algebraic group over Q. We let G := G{M.)q be 
the connected component of the identity of G'(M) so that G ~ SOo{p,q). We let 
K be the maximal compact subgroup of G stabilizing spanjca; I < a < p}. Thus 
K ~ SO(p) X SO (q) . Let D — G/Khe the symmetric space of dimension pq associated 
to G. We realize D as the space of negative gr-planes in Vr: 

(2.1) L>~{zcyR:dimz = g; (, )|, <0}. 

Thus zo = spanje,,;/:* + I < < m} is the base point of D. Furthermore, we can 
also interpret D as the space of minimal majorants for ( . ). That is, z E D defines a 
majorant ( , )2 by {x,x)z = —{x,x) if x E z and {x,x)z ~ {x,x) if a; £ z-^. We write 
( , )o for the majorant associated to the base point Zq. 

The Cartan involution Oq of G corresponding to the basepoint Zq is obtained by 
conjugation by the matrix Ip^g. We will systematically abuse notation below and write 
^o(^) for the action of the linear transformation of V with matrix Jp ^ relative to the 
above basis acting on f G V. Let g be the Lie algebra of G and t be the one of K. 
We obtain the Cartan decomposition 

(2.2) = fi©p, 

where 

(2.3) p = span{Xc,^ := A e^; 1 < a < p,p -M < < m}. 
Here w A w' e /\^ V^, is identified with an element of g via 

(2.4) {w Aw'){v) — {w,v)w' — {w',v)w. 

We let {coafj} be the dual basis of p* corresponding to {-Da^}. Finally note that we 
can identify p with the tangent space Tz^XD) at the base point Zq of D. 

Let L C y be an even Z-lattice of full rank, i.e., {x,x) G 2Z for x E L. In 
particular, L C L*, the dual lattice. We denote by r(L) the stabilizer of the lattice 
L and fix a neat subgroup F of finite index in T{L) fl G which acts trivially on L*/L. 
We let X — V\D be the locally symmetric space. We assume that X is non-compact. 
It is well known that this is the case if and only if V has an isotropic vector over Q. 
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We let r be the Witt rank of V, i.e., the dimension of a maximal totally isotropic 
subspace of V over Q. 

Let F be an isotropic subspace of V of dimension i. Then we can describe the 
^-dimensional isotropic subspace 6o{F) as follows. For U a subspace of V, let f/-*", 
resp. 1/-'-° be the orthogonal complement of U for the form ( , ), resp. ( , )o. Then 

9o{F) = (F^)^°. 

We fix a maximal totally isotropic subspace Er and choose a basis Ui,U2, ■ ■ ■ ,Ur of 
Er- Let E'j. — 9o{Ej.). We pick a basis u'^, — ' i ^ of E'^ such that {ui, u'j) — Sij. More 
generally, we let 

(2.5) Ei := spa.n{ui, . . . ,ue}, 

and we call Ei a standard totally isotropic subspace. Furthermore, we set i?^ = 
Oo^Ee) = span(M^, . . . ,u'i). Note that E'^ can be naturally identified with the dual 
space of Ef. We can assume that with respect to the standard basis of we have 

(2.6) ea = --^(m„ - u'J and e^+i-a ^ + ^")- 
for q; = We let 

(2.7) We = El/Ee, 

and note that Wg is a non-degenerate space of signature {p — i,q — i). We can realize 
We as a subspace of V by 

(2.8) We^{Ee®E',)^, 

where the orthogonal complement is either with respect to ( , ) or ( , )o. This gives a 
^o-invariant Witt splitting 

(2.9) V = Ee(BWe(B E'e- 

Note that with these choices restricts to a Cartan involution for 0{We{R)). We 
obtain a Witt basis Ui, . . . , Ue, e^+i, Cm-e, u'^, ■ ■ ■ , u[ for Vjj- We will denote coor- 
dinates with respect to the Witt basis with yi and coordinates with respect to the 
standard basis with Xi. Note that with respect to the Witt basis, the bilinear form 
( , ) has Gram matrix 



(2.10) (,) 



with J = y^ - - 

We often drop the subscript i and just write E, E', and W. 
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2.2. Parabolic Subgroups. We now describe tlie parabolic subgroups of G. We 
follow in part [4]. We let F be a flag of totally isotropic subspaces = Fq C Fi C 
F2G---GFkGF^C---cF-^cVofV over Q. Then we let P ^ Pp he the 
parabolic subgroup of G stabilizing the flag F: 

(2.11) PF-{9^G;gFi^Fi}. 

We let N_p be the unipotent radical of P. It acts trivially on all quotients of the 
flag. We let Lp — N_p\P_ and let S_p be the spht center of Lp over Q. Note that 
S_p acts by scalars on each quotient. Let M p = f^x^D{Lp) Ker(x^). We let = Np 
and L = Lp_ be their respective real points in G, and let P = P^ = (Pf(^))o) 
M = Mp = (Mp(]R))o, and A = Ap_ = (>Sp(M))o be the connected component of the 
identity in P(MX Mp(K), and Sp(U.) resp'ectively 

By conjugation, we can assume that the flag F consists of standard totally isotropic 
subspaces Ei (2.5) and call Pp a standard Q-parabolic. In that case, using the Cartan 
involution Oq, we reahze Lp (and also Sp, Mp) as ^o-stable subgroups of P: 

(2.12) Lp^pneoiP). 

Then M p is the semi-simple part of the centralizer of S^p in P. We will regularly 
drop the subscripts F, P, and P. 

We obtain the (rational) Langlands decomposition of P: 

(2.13) P ^ NAM ^ N X Ax M, 

and we write n, o, and m for their respective Lie algebras. The map P ^ N x A x M 
is equi variant with the P- action defined by 

(2.14) nam'{n, a, m) = {n Ad{am'){n), a a, mm) . 

We let F be a standard rational totally isotropic flag = Fq C Ei-^ C ■ ■ ■ C Ei^ = 
El = E and assume that the last (biggest) totally isotropic space in the flag F is 
equal to E(, for some £. 

Let Uij — span(iij^._j+i, . . . ,Uij) be the orthogonal complement of Ei._^ in Ei^ with 
respect to ( , )o and U-^ be the orthogonal complement of E'^^ in E'^.^^ and let W — 
Wi = {Eg © E'^)^. Wc obtain a refinement of the Witt decomposition of V such that 
the subspaces Ui., U^^, and W are mutually orthogonal for ( , )o and defined over Q: 

(2.15) y = (®^''] 

Then Lp is the subgroup of P that stabilizes each of the subspaces in the above 
decomposition oiV. In what follows we will describe matrices in block form relative 
to the above direct sum decomposition of V. 

We first note that we naturally have O(l^) x GL(£') C 0{V) via 




(2.16) {(%.);/ieO(Pr),yeGL(F)}, 



12 JENS FUNKE AND JOHN MILLSON 

where g = Jg*J and g* = ^g~^. In particular, we can view the corresponding Lie 
algebras o(VFm) and gl{E^) as subalgebras of g. Namely, 

(2.17) o{Wm.) ~ span{ei A ej] i < i < j < m - £}, 

(2.18) Qi{E^) ~ spanjw; A uf, i,j < i}. 

via the identification g ~ /\^ Vr. 

We let S_ be the maximal Q-split torus of G given by 

(2.19) S^{a{ti,...,tr):^i i 



We write t = {ti, . . . ,tr) and t — tJ — {tr, . . . , ii). Note 

(2.20) e^p{u\ A lii) = a(0, . . . , 0, 1, 0, . . . , 0). 

The set of simple rational roots for G with respect to P and S_ is given by A 
^{S., G) — {cci, . . . , ar}, where 

(2.21) a,(a) = (1 < i < r - 1) 

't^ if VT^ ^ 

U-ltr if H^r = 0. 



(2.22) ar{a) 



We write $(P, Ap) for the roots of P with respect to Ap_ and A{P,Ap) for the 
simple roots of P with respect to Ap, which are those a E A which act nontrivially 
on S_p. We have 

(2.23) A{P,Ap) = {a,„...,aiJ. 

Note that in the Q-split case for 0{p,p), the case E = Ep_i can never occur. 
We rather have the maximal parabolics stabilizing E = Ep = span(-ui, . . . ,Up) (with 
A(P, Ap) = {ap}) and span(Mi, . . . , Up_i, u'p) (with A(P, Ap) = {ttp-i}). 

We let Q be the standard maximal parabohc stabilizing the single totally isotropic 
rational subspace E^ of dimension £ < r. In that case, we have S^ — S_ — {a{t)} with 
a{t) = a(t, . . . ,t). 

We now consider the isotropic flag F in ^ as a flag F{E) : (0) C Ei-^ C ■ ■ ■ C Eii^ — 
E inside E. We let P' be the parabohc subgroup of GL(P) stabihzing F{E). Then 
for the real points P' — (P^(IR))o, we have 

(2.24) P' = Np>AMp,, 
with unipotent radical Npi and Levi factor 

k 

(2.25) Mp, = J]SL(^7,,(M)). 

Here A is as above, viewed as a subgroup of GL_,_(E]r). Note Mqi ~ SL(Ek). Fur- 
thermore, we can view P' and its subgroups naturally as subgroups of of P via the 
embbeding of GL(£') into 0(\^) given by (2.16). 
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Returning to P, we have 

(2.26) Lc^U'h^y,he SO{W),g = diag(^i, . . . , ^fc) e J] GL{Ui.), 

Thus 
(2.27) 

We also define 
(2.28) 

and we write 
(2.29) 

where ps — si{E) n p and 

(2.30) pw — Ow f^P — span{Dafj. = Ca A e^; £ + 1 < a < p, p + 1 < /j, < m — i}. 
We naturally view iVp, C SL(ii^) as a subgroup of the unipotent radical N_p via 



M ~ SOo(iy]R) X Mp,. 
pM = p n m, 

PM ^PW® PE, 



(2.3i; 



n' ^ N{n') :-- 



EN p. 



n' 



We then have a semidirect product decomposition 
(2.32) Np^Np^x Nq, 

where Q is as above the maximal parabohc that stabilizes E. Furthermore, we let 
Z_Q be the center of Njq C N_p. It is given by 



(2.33) 



'1 b 

z{b) := ( 1 1 ; J*bJ = -b 



> . 



Then for the coset space N_p/{N_p, K Z_q), we have 

(2.34) Np/{Np^ K Zq) ^ ATq/Zq c^W®E 

as vector spaces. Explicitly, the basis of E gives rise to an isomorphism M/® £^ ~ W^. 
Then for (wi, . . . , W() e 1^^, the corresponding coset is represented by 

(h i-^wi) -wl\ 



(2.35) 



Iw -Wi . . . -Wi 

If 



Here we write wf = ^{wi,Wi) for short. 

On the Lie algebra level, we let 3q be the center of Uq C Up, whence corresponding 
to (2.33) 

(2.36) 3Q~/\'^M. 
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We let Up/ be the Lie algebra of iVp/; thus Up/ C E'^ Ai^R = gl{E]^). Corresponding to 
(2.35), we can reahze (8) E'm as a subspace of n. Namely, we obtain an embedding 

(2.37) Wm(^Eu^ n, 

(2.38) w (S> u ^ w A u —: nu{w) , 

and we denote this subspace by riv^, which we frequently identify with <S) E-g.- 
Furthermore, this embedding is o(VFm) ® 0((£']K)-equivariant, i.e., 

(2.39) [X, riuiw)] = riuiXw) [Y, n„(w)] = nmiw) 
for X e o(1V]r) and Y G QliE^). We easily see 

(2.40) exp(n„,(w)) = n(0,...,w,...,0). 
A standard basis of nw is given by 

(2.41) X^i := n„, (e^) = e^, A Ui, X^i := (e^) ^e^AUi 

with l<i<£, l-\-l<a<p, and p + l<ijL<m — i. The dual space we can 
identify with Wr E^, and we denote the elements of the corresponding dual basis 

by i^ai = eaAu[ and u^i = -e^ A u[. 
Summarizing, we obtain 

Lemma 2.1. We have a direct sum decomposition (of vector spaces) 

Tip = tip' ® nw ® 3q- 

Furthermore, the adjoint action o/o(1Vr) © 0t(-E'R) on rip induces an action on the 
space np/(np/ ®3q) — xiw such that 

nw '^Wm® Em 

as o{Wm,) © Ql{Eg) -representations. Finally, the action of A on has weights 
q;~^, . . . , with respective weight spaces (8) C//^ , . . . , (8) C/^'^ . 

2.3. The Maurer Cartan forms and horospherical coordinates. The Lang- 
lands decomposition of P gives rise to the (rational) horospherical coordinates on D 
associated to P by 

(2.42) a^ap-.NxAxDp — >D 
given by 

(2.43) a{n,a,m) = namzQ. 
Here 

(2.44) Dp = Mp/Kp 

is the boundary symmetric space associated to P. Here Kp — M (1 K. 

We note that the boundary symmetric space Dp always factors into a product of 
symmetric spaces for special linear groups and one orthogonal factor, namely, the 
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symmetric space associated to SO(H^). We call the associated symmetric space Dw 
the orthogonal factor in the boundary symmetric space Dp. We have 

k 

(2.45) Dp^Dwxl[Du,., 

where Du. denotes the symmetric space associated to SL(C/j.). 

We now describe how the basic cotangent vectors cUq,^ = (e^ A e^)* G p* ~ T*^{D) 
look like in NAM coordinates. We extend a to NxAxMxK — > G by 
a{n, a, m, k) = namk, and this induces an isomorphism between the left-invariant 
forms on NAM (which we identify with n*®a*®p^) and the horizontal left- invariant 
forms on G (which we identify with p*). Thus we have an isomorphism 

(2.46) a* :p* — ^ n* a* © p;^. 
Explicitly, we have 

Lemma 2.2. Let 1 < i < i. For the preimage under a* of the elements in coming 
from ® E, we have 

(2.47) a* OJam+l-i — ~:^^ai, 

where i + 1 < a < p. Furthermore, for the ones coming from VF_ <S> E, we have 

(2.48) a*ujn, = j^iy^i, 

where j9+l</i<m+l — On p\^, the map a* is the identity. In particular, for 
£ + 1 < a < p and any ii > p + 1, we have 

(2.49) (7*a;«^ G p^ ® n^^. 

The remaining elements of p* are of the form with p+l<ij,<m + l — £. These 
elements are mapped under a* to n*p, © a* © p^ C qI{Er)* . 

2.4. Borel-Serre Compactification. We now describe the Borel-Serre compactifi- 
cation of D and of X = r\D. We follow [4], 111.9. We first partially compactify the 
symmetric space D. For any rational parabolic P, we define the boundary component 

(2.50) e(P) ^NpxDpc^ P/ApKp. 

Then as a set the (rational) Borel-Serre enlargement D — D is given by 



(2.51) D = DU JJe(P), 



p 



where P runs over all rational parabolic subgroups of G. As for the topology of D, 
we first note that D and e(P) have the natural topology. Furthermore, a sequence 
of yj = ap{nj, ttj, Zj) & D in horospherical coordinates of D converges to a point 
{n, z) e e(P) if and only if uj — * n, Zj — > z and Oi{aj) oo for all roots a e $(P, Ap). 
For convergence within boundary components, see [4], III. 9. 
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With this, D has a canonical structure of a real analytic manifold with corners. 
Moreover, the action of G{Q) extends smoothly to D. The action oi g — kp — 
kman e KM AN — G on e(P) is given by 

(2.52) g ■ (n', z') ^ k ■ (Ad(am)(nn'),mz') e e(Ad(k)P) = e(Ad(g)P) 
with k ■ {n',z') = {Ad{k)n, Ad{k)mKAd{k)p) e e{Ad{k)P). Finally, 

(2.53) X := r\D 

is the Borel-Serre compactification oi X — r\D to a manifold with corners. If 
P.i:---:P.k is a set of representatives of F-conjugacy classes of rational parabolic 
subgroups of G, then 

k 

(2.54) r\D = T\DuY[TpME.)i, 

i=l 

with Fp. = F n Pi. We will write e'(P) = Fp\e(P). We write Tm for the image of Fp 
under the quotient map P — > P/N. Furthermore, we Fp acts on the quotient£']^/£'K, 
and we denote this transformation group by Tw- Note that Fm and Fw when viewed 
as subgroups of P contain FflM and FnSOo(H^M) respectively as subgroups of finite 
index. 

We now describe Siegel sets. For t e 1R+, let 

(2.55) Ap^t = {a e Ap; a{a) > t for all a e A(P, Ap)}, 
and for bounded sets U C Np and V C Dp, we define the Siegel set 

(2.56) &p,u,t,v xAp^txV cNpxApX Dp. 

Note that for t sufficiently large, two Siegel sets for different parabohc subgroups are 
disjoint. Furthermore, if Pi, . . . ,Pk are representatives of the G(Q)-conjugacy classes 
of rational parabolic subgroups of G, then there are Siegel sets Si associated to Pj 
such that the union [J 7r(5j) is a fundamental set for F. Here tt denotes the projection 
TT : P> ^ F\P>. 

3. Review of representation theory for general linear and 

orthogonal groups 

In this section, we will briefly review the construction of the irreducible flnite 
dimensional (polynomial) representations of GL(C") and 0(V"). Here, in this section, 
we assume that V is complex space of dimension m. Basic references are [9], §4.2 and 
§6.1, [13], §9.3.1-9.3.4 and [2], Ch. V, §5 to which we refer for details. 

3.1. Representations of GL„(C). Let A = (6i, 62, • ■ ■ : K) be a partition of i'. We 

assume that the 6j's are arranged in decreasing order. We will use D{X) to denote the 
Young diagram associated to A. We will identify the partition A with the dominant 
weight A for GL{n) in the usual way. A standard filling A of the Young diagram D{X) 
by the elements of the set [^] = {1, 2, ■ ■ ■ , f } is an assignment of each of the numbers 
in [£'] to a box of D{X) so that the entries in each row strictly increase when read 
from left to right and the entries in each column strictly increase when read from top 
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to bottom. A Young diagram equipped with a standard filling will be also called a 
standard tableau. 

We let s(t(A)) be the idcmpotent in the group algebra of the symmetric group 
S^i associated to a standard tableau T with i' boxes corresponding to a standard 
filling t{\) of a Young diagram D{X). Note that Spj acts on the space of ^'-tensors 
T^'(C") in the natural fashion on the factors of T^'(C"). Therefore s{t{\)) gives rise 
to a projection operator in End{T^' {C^)) , which by shght abuse of notation we also 
denote by s{t{\)). We write 

(3.1) §,(,)(n = .(t(A))(r^'(n). 

We have a direct sum decomposition 

(3.2) r^'(n = 00St(A)(n, 

A t(A) 

where A runs over all partitions of I' and t{\) over all standard fillings of -D(A). 
This gives the decomposition of (C") into irreducible constituents, i.e, for every 
standard filling t(A), the G'L(C")-module St(A)(C") is irreducible with highest weight 
A. In particular, E>t{\){'C^) and St/(A)(C") are isomorphic for two different standard 
fillings t{\) and t'{\). We denote this isomorphism class by §a(C'^) (or if we do not 
want to specify the standard filling). 

Explicitly, we let A be the standard filling of a Young diagram D{A) corresponding 
to the partition A with less than or equal to n rows and boxes by 1, 2, • • • , obtained 
by fiUing the rows in order beginning at the top with 1, 2, • • • , We let R{A) be 
the subgroup of S^t which preserves the rows of A and C{A) be the subgroup that 
preserves the columns of A. We define elements r{A) and c{A) by 

(3.3) r{A) = ^ s and c{A) = ^ sgn(s)s. 

Let h{A) be the product of the hook lengths of the boxes in D{A), see [9], page 50. 
Then the idempotent s{A) is given 

(3.4) s{A) = -1^^c{A)t{A). 
We will also need the "dual" idempotent ^(^4)* given by 

(3.5) s{Ay = -l^^r{A)c{A). 

We let £i, . . . , denote the standard basis of C" and 9i, . . . ,9n G (C")* be its dual 
basis. We set 

(3.6) £^ = eji . . . ® 

and let 9a be the corresponding element in T^'(C")*. Then s{A){eA) is a highest 
weight vector in Sa(C"), see [13], §9.3.1. We have 

Lemma 3.1. 

s{Ar94s{A)eA) = 
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Here \R{A)\ is the order of R{A). 
Proof. We compute 

s{Ay9A{s{A)eA) = 9A{s{AfeA) = 9A{s{A)eA) = 



\R{A)\ 



9A{c{A)eA) = 



\R{A)\ 



9A{eA). 



h{A) ' ' ^' h{A) 

The last equation holds because 9a{(ISa) = for any nontrivial q in the column group 
of A as the reader will easily verify. We have used r{A)eA = \R{A)\eA (since all row 
permutations fix ea) and 5(^4) = j^c{A)r{A). □ 

3.2. Enlarging the Young diagram. The following will be important later. 

We let B — Bn^£ be the standard tableau with underlying shape D{B) an n by £ 
rectangle with the standard filling obtained by putting 1 through i in the first row, 

£+1 through 2£ in the second row etc. Then D{B) is the Young diagram corresponding 
to the dominant weight Iwn- Here tu„ = (1, 1, ■ ■ ■ ,1) is the n-th fundamental weight 
for GL{n). Wc note that we have Sb = ^i® ■ ■ ■ ® ^nd 6b = 9[® ■ ■ ■ ® 9^. 

Lemma 3.2. (1) s{B)T"'^{C"') is one- dimensional and 

s(B)r'*^(C") = Cs{B)eB 

as GL{n,C) -modules. 
(2) s{B)*T"'^[C"')* is one- dimensional and 

s{ByT''\c'y = cs{By9B 

as GL{n,C) -modules. In particular, we have 



s(5)*T"^(C")* = (/\"(C")* 



We let A be the standard filling of the Young diagram D{X) as above. Then B\A 
denotes the standard tableau with underlying shape D{B\A) given by making the 
shape of A abut B (on the right), using the above filling for B and filhng A in the 
standard way (as above) with ni -\- 1 through ni-\- £'. For example, if 



B 



1 


2 


3 


4 


5 


6 


7 


8 


9 



and A 



1 


2 


3 


4 


5 







1 


2 


3 


10 


11 


12 


then B\A^ 


4 


5 


6 


13 


14 






7 


8 


9 









We have an idempotent s{B\A) in the group ring of Sn£+£' and the nl + £' ten- 
sor eB\A £ T"^+^ (C"), which gives rise to a highest weight vector s{B\A)6b\a in 
s(5|A)(T"^+^'(C")). Note 

(3.7) eB®eA = eB\A- 

Lemma 3.3. There is a positive number c{A, B) such that 

s{B)eB ® s{A)eA = c{A, B)s{B\A)sb\a- 

Proof. Since the Young diagrams D{B) and D{A) are abutted along their vertical 

borders, we see 



(3.8) 



c{B\A) = {c{B) 



o (1„^ c{A)) = {Ine c{A)) o {c{B) (g) le) 
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Also (for any standard tableau C) 

(3.9) riC)ec = \RiC)\Ec. 

Then we compute (using the three equations (3.7), (3. 8), (3. 9)) 

s{B)eB ® s{A)sA = j^c{B)r{B)eB ® -^c{A)r{A)eA 
^ 'c{B) ® V) o (1„^ ® c{A)){{r{B)eB ® r{A)eA) 



h{B)h{A) 

\R^B)\R{A)\ |i?(i?)|i?(A)| 
^ h{B)h{A) <B\A){eB®eA) - <B\A)eB\A 

l^(^)l^(^)l 1 .oMV /^(g|A) \R{B)\\Rm 

□ 

Corollary 3.4. Under the identification o/T"^(C") ® T^'(C") ^ t^'W' {C^) given by 
tensor multiplication, we have the equality of maps 

s(B)0s(A) = s(B\A). 

That is, 

§b(C")«)§a(C")=§b|a(C") 

as (physical) subspaces of T'^^'^^' (C^) . The same statements hold for the dual space 
§^I^(C'*^+^')* etc. 

Proof. Since Sb(C"') is one-dimensional, the tensor product §s(C"') ® §a(C"') de- 
fines an irreducible representation for GL„(C") (under the tensor multiplication map 
T"^(C'") (8)T^'(C") inside T"^+^'(C")). But by Lemma 3.3 it has nonzero intersection 
with the irreducible GL„(C)-representation Sb|a(C") inside J'"^+^'((C"). Hence the 
two subspaces coincide. □ 

3.3. Representations of 0(F). We extend the bilinear form ( , ) on y to T^' (V) as 

the £'-fold tensor product and note that the action of S^r on {V) is by isometrics. 
We let \/[^'] be the space of harmonic ^'-tensors (which are those f'-tensors which 
are annihilated by all contractions with the form ( , )). Wc let Ti. be the orthogonal 
projection Ti, : T^' (V) — >■ V^^'^ onto the harmonic ^'-tensors of V. Note that the space 
of harmonic ^'-tensors is invariant under the action of S'^. We then define for A as 
above the harmonic Schur functor S[t(A)] (V) by 

(3.10) SitW]{V)^mtix){V). 

If the sum of the lengths of the first two columns of -D(A) is at most m, then S[t(A)] (Vc) 
is a nonzero irreducible representation for 0(Vc), see [9] section 19.5. Otherwise, it 
vanishes. Of course, for different fillings t{X) of -D(A), these representations are all 
isomorphic and we write S[a](V") for the isomorphism class. Furthermore, it is also 
irreducible when restricted to G unless m is even and i{X) — in which case it splits 
into two irreducible representations. If i{X) < [y], then the corresponding highest 

weight A for the representation §[a](V^) of G ihas the same nonzero entries as A. 
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4. The Weil representation 



We review different models of the Weil representation. In this section, V denotes 
a real quadratic space of signature (p, q) and dimension m. 

We let V be a real symplectic space of dimension 2n. We denote by G' = Mp(n, M) 
the mctaplectic cover of the symplectic group Sp(y ) = Sp(n, M) and let g' be its Lie 
algebra. We let K' be the inverse image of the standard maximal compact U(n) C 
Sp(n, M) under the covering map Mp(n, M) — > Sp(n, M). Note that K' admits a 
character det^^^, i.e., its square descends to the determinant character of U(n). The 
embedding of U(n) into Sp(n, M) is given hj A + iB {j^ba)- 

We write Wn,y for (an abstract model of) the A''-finitc vectors of the restriction of 
the Weil representation of MpiV' (8) V) to Mp(n, K.) x 0{V) associated to the additive 
character t ^ e^'^**. 

4.1. The Schrodinger model. We let V( be a Langrangian subspacc of V. Then 
V V( is a Langrangian subspace of V ^ V (which is naturally a symplectic space 
of dimension 2nm). The Schrodinger model of the Weil representation consists of the 
space of (complex- valued) Schwartz functions on the Langrangian subspace Vl®V ■2:^ 
V^. We write »S(y") for the space of Schwartz functions on and write u — ojny 
for the action. 

The Siegel parabolic P' = M'N' has Levi factor 



It is well known that we can embed P' into Mp(n, R), and the action of P' on SiV"^) 
is given by 



(4.1) 




and unipotent radical 



(4.2) 





u; (m'(a)) ^(x) = (det a)"*/V(xa) 
uj {n'{h)) ip{^) = e'^^*^(''('''^)V(x) 
. . , Xn) e y". The central acts by 



(deta > 0), 



(4.5) 




if m is odd 
if m is even 
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We let be the space of smooth, i.e., i^'-finite, vectors inside the space 

of Schwartz functions on V^. It consists of those Schwartz functions of the form 
p(x)(/7o(x), where p is a polynomial function on V". 

4.2. The mixed model and the definition of local restriction for the Weil 
representation. We now describe a different model for the Weil representation, 
the so-called mixed model. Furthermore, we will define a " local" restriction from 
S{V"') to the space of Schwartz functions S{W"') for W, a (real) subspace of signature 
{p-i,q-i). 

4.2.1. The mixed model. We let E = he one of the standard totally isotropic 
subspaces of V, see (2.5). As before, wc identify the dual space of Ei with E'^. 

Accordingly, for the decomposition V = E (BW (B E' , we write x = j for x G V"', 

where u e E"", u' G {E')"^, and G W". We then have an isomorphism of two 
models of the Weil representation given by 

(4.8) SiV") 5((E')")®'5(1^")®5((E')") 

ip\ — >^ 

given by the partial Fourier transform operator 















[u'j 







(4.9) ^ ^wj = I ^\^w\ e-''''''-^'''^>du 

with ^,u' G (E')" and e W^"- 

We will need some formulae relating the action of u; in the two models. We have 

Lemma 4.1. Let (^xi j e{E'®W® E'Y . 

(i) Let n G Nq and write n{u')w for the image of n{u') under the orthogonal 
projection onto W Then 

n^Ci^, xiy, u')) = e (tr(n(x^y + m')> 0) 'fCi^, + n{u')w, u')). 

(ii) For g G SL(£^) G G (in particular, g G Npr or g E Mpi ) we have 

^(*(C,xh/,m')) = ^{\gi,^w,gu')) 

with g = Jg*J and g* = ^g^^. 

(iii) For t = (ti, . . . ,ti), set t = tJ — {ti, . . . ,ti) and |t| — ti ■ t2 ■ ■ - t^. Then for 
a{t), we have 

^^typC{^,xw,u')) = |tr^(^(te,x^^, W))- 

(iv) For h G SOo(Vl^) C M, we have 

(v) For m'{a) = ( i ) G M' C Sp{n, R) with a G GL+{R), 

{m'{a)(p)C{^,xw,u')) = (deta)t-^^(*(^a*,xvya,'u'a)) 
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(vi) Forn'ih) = {H) e N' C Sp{n,M) with b e Sym„(R), 

Proof. This is straightforward. □ 
We obtain 

Proposition 4.2. Let (p e S{V'"'). Then the restriction of (p to W^, 

defines a G' x MN intertwiner from to S{W^). Here, we identify W with 

E-^/E to define the action of MN on W. In particular, N and Mpi (see 2.27) act 
trivially on SiW"^). 

4.2.2. Weil representation restriction. 

Definition 4.3. Let (p G S{y"') and let P a standard paraboUc of G, and let E = Ei 
be the biggest totally isotropic subspace of V in the flag stabilized by P with Witt 
decomposition V — E ®W ® E'. We then define the "local" restriction r^{(p) e 
<S(1^") with respect to P for the Schrodinger model of the Weil representation W by 

r^{ip) = (p\w^. 

We now describe this restriction on a certain class of Schwartz functions on V^. 

XlJ 



For X = {xi, . . . , Xn) e V^, we write I : 1 for the standard coordinates of Xj. We 

define a family of commuting differential operators on by 

1 d 



(4.10) Hrj = Xrj - 



2tt dXrj 



where 1 < r < m and 1 < j <n. Then there exists a polynomial such that 

(4.11) H,>o(x) = ^,.(x,,)vPo(x), 

where v^o(x) is the standard Gaussian, see (4.7). In fact, it is easy to see that Hk is 
essentially the fc-th Hermite polynomial Hk{x) = (— l)*^e^^^e~^^: 

(4.12) Hk{x) = {2T:)-^/^Hk (V2^x) . 

We let A e M^xn(^) = {^rj) be an integral matrix with non-negative entries and split 
A into A+ e Mpxn(Zi) and A_ e Mgxn(Z) into its "positive" and "negative" part, 
where A_|_ consists of the first p rows of A and A_ of the last q. (Recall m — p + q). 
We define operators 



l<r<m l<a<P p+l</i<m 

l<j<n 



SO that ?i A = 'Ha+'Ha. . Here again we make use of our convention to use early Greek 
letters for the "positive" indices of V and late ones for the "negative" indices. 
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Definition 4.4. For A as above, we define the Schwartz function Lp^ by 

A (x) = H/^ipo (x) = JJ Hs^. {Xaj ) Hs^^ {X^j ) (x) . 

l<a<p 
p+l<(U<m 
l<j<n 

We now describe (p^ in the mixed model. The superscript V emphasizes that 
the Schwartz function is associated to the space V. We begin with some auxiliary 
considerations. The following little fact will be crucial for us. 

Lemma 4.5. For a Schwartz function f e S{M.), let /(^) = J^f{y)e~^'^^^^dy be its 
Fourier transform. Let gk{y) — Hk{—^^^~^^'^- Then 



Proof. We use induction and differentiate the equation {gk){—y) = Hk{^)e '^^^ . The 
assertion follows from the recursion Hk+i{y) = 2yHk{y) — ^H'f^{y) , which is immediate 
from the definition of H^. The claim also follows easily from [21], (4.11.4). □ 

Remark 4.6. Recall that on the other hand that Hk{y)e^^y^ is an eigenfunction 
under the Fourier transform with eigenvalue {—iY^ see [21], (4.12.3). This fact is 
underlying the automorphic properties of the theta series associated to the special 
Schwartz forms ^nq,[\]- 

The Gaussian is given in standard coordinates by V'^(x) = exp(— vr ^"=1 'YlT=i ^Ij)- 
In Witt coordinates, we have Xrj = -^{yrj - y(m-r)j) and x^^_r)j = -^{Vrj + y{m-r)j); 
thus + xl^_^)^. = yl^ + yl^_^^^ for r < £. Thus 

(4-13) ifil ^ ^ Yl^y'^rj + ylm-r)o)^ {^w) ■ 

We write shghtly abusing 

1 = exp l^-TT J2 J2iyrj + ylm-r)j) 

We let A' be the truncated matrix of size (m — 2f) x n given by eliminating the 
first and the last I rows from A. We let A" be the matrix of these eliminated rows. 
Note that TYa' now defines an operator on SiyV^) and TYa" on 5((£^ ©£")"). We also 
obtain matrices A'_,_ of size {p — t) x n and A'_ of size {q — i) x n by ehminating the 
first i and the last i rows from A+ and A_ respectively. Similarly we obtain A" and 
A'^. With these notations we obtain 

Lemma 4.7. (i) 

<^A I I ^ <PA'{^w)<pi"{tu'). 

u' 




24 JENS FUNKE AND JOHN MILLSON 

r]^ (V^K) (xp^) = ^^,(xp^Vf„(0,0). 

In our applications all entries of A_ will be zero, so A = A_|_ (by abuse of notation). 
We first note 

Lemma 4.8. 

\i=l a=l ) 

In particular, if in addition all entries of A'^ vanish, then 

Proof. This follows from applying Lemma 4.5. □ 
We conclude 

Proposition 4.9. (i) Assume that one of the entries of A'_|_ is nonzero, then 

<(v^Kj = 0. 

(ii) // all of the entries of A'_[ vanish, then 

(,.v \ _ w 

Remcirk 4.10. Of course the analogous result holds for rp^{(p^_). However, a general 

formula for the restriction of r^{(p^), i.e., for (f^„{0, 0), is more complicated (and is 
not needed in this paper). 

4.3. The Fock model. It will be also convenient to consider the Fock model JF = 
J-'n V of the Weil representation. For more details for what follows, see the appendix 
of [11]. 

There is an intertwining map i : SiV) p(C"(P+«)) from the X'-finite Schwartz 
functions to the infinitesimal Fock model of the Weil representation acting on the 
space of complex polynomials P(C"*^^^^'') in n(p + q) variables such that i{'^o) = 1. 
We denote the variables in ^(C"*^^"'"'^^) by Zai (1 < a < p) and z^i (p + 1 < < p + ?) 
with i — 1, . . . ,n. Moreover, the intertwining map i satisfies 

1 a \ _i 1 / 1 9 \ _i 1 



By slight abuse of notation, we use the same symbol for corresponding objects in the 
Schrodingcr and Fock model. 

In the Fock model, if^ looks as follows. 

Lemma 4.11. 

p+l</i<m 
l<j<n 
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Proposition 4.9 translates to 
Proposition 4.12. // one of the entries o/A'_|_ is nonzero, then 

// all of the entries of A'i, vanish, then 



e+l<a<p 
l<j<n 

5. Differential graded algebras associated to the Weil 

representation 

In this section, we construct the differential graded algebras Cy and Ap, which 
were introduced in the introduction and define the the local restriction map rp from 
Cy to A'p. Again V will denote a nondegenerate real quadratic space of dimension 
m and signature (p, g). 

5.1. Relative Lie algebra complexes. For convenience of the reader, we very 
briefly review some basic facts about relative Lie algebra complexes, see e.g., [6]. 
For this subsection, we deviate from the notation of the paper and let Q be any real 
Lie algebra g and let t be any subalgebra. We let (C/, tt) be a representation of Q. We 
set 

(5.1) C'^(g,t;U) ^ [Rom f\\g/t), U)]' ^[/\\g/tr0u]\ 

Aq 
{g/t) is induced by the adjoint representation. Thus 

Ci{g, 6; U) consists of those ip e Hom /\\g/i), C/) such that 

q 

(5.2) ^<^(Xi, ...,[X,Xl...,Xg)^X- (^(Xi, ...,X,) {Xe fi). 

i=l 

The differential d : ^ 0"+^ is defined by 

<? 

(5.3) d^{Xo, Xi, . . . , Xg) = ^(-1)*X, • ^{Xo, ...,Xi,...,Xg) 

+ 5](-l)^+V([^i,^i],^o, ...,Xi,...,Xj...,Xg) 



i<j 



for Xo, . . . ,Xq e g/t. We let {Xi} be a basis of g/t and let {uji} be the dual basis. 
Then in the setting of A (g/t)* ® ^7 , the differential d is given by 

(5.4) d^Yl ^(^^) ® ^(^^) + Iy1 ^('^^) ^^*(^^) ® 1- 
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Here A(uJi) denotes the left multiplication with Ui in ^ id/i)*, and ad*(X) is 
the dual of the adjoint action on yAy (0/6)*, that is, {a,d*{X){a)){Yi,- ■ ■ ,Yq) — 
ELi ...AY,,Xi...,Y,).We easily see 

Lemma 5.1. Consider two relative Lie algebra complexes C'{q, fi; U) and C*{g', t'; U'). 

Then the following datum, 

(i) p : — >• 0', a Lie algebra homomorphism such that p{t) C t' , 

(ii) T : U' ^ U , an intertwining map with respect to p (i.e., T{p{X)-u') = X-T{u') 
for X e q), 

induces a natural map of complexes 

C'(Q',t';U')^C'(Q,i;U) 

given by 

(/? I— > T o o p. 

r .q -if 

When realizing (p as an element A {q' /t')* <SiU'\ , then the map is given by 

p*®T, 

where p* : (g'/^')* ~^ id/^)* '^^ the dual map. 

Now let G be any real Lie group with Lie algebra Q and let K be a closed connected 
subgroup of G (not necessarily compact) with Lie algebra t. For U a smooth G- 
module, we let A'^{G/K;U) be the [/-valued differential g'-forms on G/K (with the 
usual exterior differentiation). The G-action on A'^{G/K; U) is given by 

(5.5) {g o wUX) = gicjg-u^ig-^ ■ X)), 

for u e A''{G/K;U), x e G/K, and X e T^{G/K). Then evaluation at the base 
point oi G/K gives rise to an isomorphism of complexes 

(5.6) A'iG/K;Uf c^G'iQ,t;U) 
of the G-invariant forms on G/K with G*{g, fi; U). 

5.2. The differential graded algebra Cy. We begin this section by defining a 
differential graded (but not graded-commutative) algebra Gy. We first define the 
underlying complex. 

The complex Cy is essentially the relative Lie algebra complex for 0{V) with vahies 
in Wn,y tensored with the tensor algebra of Vc and twisted by some factors associated 
to C". Precisely, it is the complex given by 



Here j, r, k are nonnegative integers and A^{D) denotes the space of complex- valued 
differential r-forms on D. We let U — /\"(C"')*, and we define the action of K' on 
T*(f/)[2^] by requiring K' to act by the character det""*^^" on T^{U). Thus K' acts 

p — q 

by algebra homomorphisms shifted by the character det 2 . We will usually drop the 
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[^] in what follows. Also note that all tensor products are over C. The differential 
is the usual relative Lie algebra differential for the action of 0{V). The group K' acts 
on the first three factors, while the maximal compact subgroup Ky — K oi SOo{V) 
fixing the bascpoint zq acts on the last three factors. Finally, the symmetric group 
Sk acts on the second and the last factor. 

We now give the complex Cy an associative multiplication. In order to give the 
complex the structure of a graded algebra we choose as a model for the Weil represen- 
tation that has an algebra structure, the Fock model J-'ny, the multiplication law is 
multiplication of polynomials. However, it is important to observe that K' does not 
act on J-n,v by algebra homomorphisms (but rather by homomorphisms twisted by 

p—q . 

the character det 2 ). Now the vector space underlying Cy is a subspace (of invari- 
ants under a group action) of a tensor product of graded algebras. Thus it remains 
to prove that the group acts by homomorphisms of the product multiplication. 

Lemma 5.2. The group K' x K x Sk acts by algebra homomorphisms on the tensor 
product of algebras T'{U) ® r*(C")* ® Wn,v /\' p£ ^•(^c)- 

Proof. The statement is obvious except possibly for the action of the group K'. The 
group K' acts on the algebra J-n,v by algebra homomorphisms twisted by the character 

det^~. It acts on the tensor product T'{U) by algebra homomorphisms twisted by 

p—q 

the inverse character det 2 . The two twists cancel on the tensor product and we 
find that K' acts by algebra homomorphisms. □ 

Sometimes it is more convenient to view an element (p e C^f'^ as an element in 

(5.7) [Hom (t\C^);T^{U) Wn,v ® /\' P*c® T''(Vc))Y '''''''' ' 

For w G T'^(C"), we write ip{w) for its value in T^{U) ® Wn,v ® /\^ Pc® T^{Vc). 
By Schur-Weyl theory, see [9], Lecture 6, we have the decomposition 

(5.8) T''{Cy ~ 0s(t(A))(r'=(C")*) ® V* 

A 



Here the sum is over the Young diagrams A with k boxes and no more than n rows, 
t(A) is a chosen standard filling of A for each A and V\ is the irreducible representation 
of Sk corresponding to A. We also have the corresponding decomposition 

(5.9) T\Vc) ^ ^s{t'i^i)){T\Vc)) V,. 

Combining the two decompositions we obtain 
(5.10) 

ci}'''' ~ [t^{u) ® §t(A) (c")* ® Wn,v A' « St'(^) (Vc) ® v^] '''''''' . 

Noting that 

(5.11) (i^;®v;)^'=- 
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wc obtain 
Lemma 5.3. 



X 



K'xK 



We have assumed (as we may do) that the filhngs t(A) and t'{X) are the same. For 
the summands in the lemma we write C^y'*^^^ (or just C^y'^ if we do not want to 
specify the fiUing) and obtain the complex Cy^ introduced in the introduction. The 
application of the Schur functor S*^p^^(-) on T'*'(C")* or equivalently applying St(;^)(-) 
on T^(yc), gives rise to a projection map 

(5.12) 7r,(A) : C'/^' C'/''^'\ 
That is, 

(5.13) 7rt(A) = 1[/ (8) s(i(A))(c»)* <8) lw„,v ® h* ® 

= 1[/ (g) lc« lw„,v ® Ip* ® s(t{X))v 

Here we have used subscripts to indicate which spaces the respective identity transfor- 
mations 1 operate on. We will do this henceforth. We apply the harmonic projection 
Hv, see (3.10), on the last factor to obtain S[t(A)] (Vc), and we obtain a complex Cy'**-^^' 
(or Cy^^^) and a projection map 

(5.14) 7r[,(,)] : 0^}^'' C^>'[*W1. 
That is, 

(5.15) n[t(x)] = 1[/ ® Ic" «) lw„,^ «) Ip* ® s{[t{X)])v 

= (l{7 <S) Ic" <S) lwn,v ® Ip* ® "^v) o 7rt(A). 

Remark 5.4. We can interpret an element (p e C^'^''^ as a x X x jSj^-invariant 
homomorphism from r'=(C") to T^{U) ® >Vn,y ® /\''pc ® T'=(l^c), see (5.7). In this 
setting, we can interpret Trt(x)(p as the restriction of the homomorphism (f to the 
St{A)(C"'). From this point of view, Lemma 5.3 states that the homomorphism TTt{\)V 
on Sj(A)(C") automatically takes values in Wny ® A^Pc ® St(A)(Vc). 

5.3. The face differential graded algebra A'p and the map rp. In this section 
we assume P is the stabilizer of a standard flag Ei-^ C Ei^ C ■ ■ ■ C Ei^, — Ei and 
Np is the unipotent radical of P. We will abbreviate Ei^, to E and let Q be the 

stabilizer of E (a maximal parabolic subgroup). We will now construct a differential 
graded algebra Ap, which is the relative Lie algebra version of a differential graded 
subalgebra of the de Rham complex of the face e(P) of the Borel-Serre enlargement 
of D. We will continue with the notation of section 2. 

We define the differential graded algebra A*p associated to the face e(P) of the 
Borel-Serre boundary corresponding to P by 

(5.16) A^/''^ = [r^ (C/) (8) T\cy ® Wn,w ® /\\n Pm)c ® T\Vc)\ 
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Furthermore, we define and Ap as for Cy. 

Definition 5.5. Tlie "local" restriction map of de Rham algebras with coefficients 

rp:C'y^ A'p 

of de Rham algebras with coefficients is given by 

1 (8) 1 (g) (g) t* (g) 1, 

where 

i : n ® m 

is the underlying Lie algebra homomorphism, and the map from the coefficients of 
Cy to the coefficients of Ap is given by the tensor product 

1 (8) 1 ® rP' ® 1, 

where r'p : Wny y^n,w is the restriction map of the Weil representation (see 
Definition 4.3). By Lemma 5.1 we therefore see that rp is a map of complexes. We 
note that is not a ring homomorphism so rp is not a map of algebras. Since rp 
commutes with the action of the symmetric group Sk, we obtain maps Cy —>■ A*p 
and Cy'^' A'p'^'^ as well, which we also denote by rp. 

Note that the induced map l* : {g/t)* ~ p* — >■ ((n © m)/tM)* — (n © pu)* is 
the composition of the isomorphism a* : p* — > (n © a © Pm)*, see (2.46), with the 
restriction (n © o © Pm)* ^ (n © Pm)*- 

Finally observe that on the level of homogeneous spaces, the map rp arises by 
realizing e(P) as the orbit of the basepoint zq under the group NM. So in this 
setting, we are no longer thinking of e(P) as being at the boundary of D; we have 
pushed e(P) far inside D. 

6. Aspects of the Cohomology of np and the map lp 

6.1. Cohomology of the nilpotent. In what follows, we discuss some aspects of 
the Lie algebra cohomology of the nilpotent Lie algabra Up which we need later. 

As before, we let P be a standard parabolic subgroup of G. Recall (see section 
2) that we have the decomposition of vector spaces rip = npr © Uq, where Q is the 
maximal parabolic containing P. For the two-step nilpotent algebra Uq, wc have the 

central extension 3q ^ Uq ^ nw with }q c::^ ^ E and xxw — W <^ E. On the other 

hand. Up/ is a nilpotent subgroup of sl{E) d E' ® E. 

We assume for the next subsections that V, W, E, £", np etc. are defined over C. 
We let 

C^' = f\{np)®T^'{V) 

be the complex for the nilpotent cohomology with coefficients in T^' (V) and define 
analogously C^'^ and C''^' for Sa(T^) and §[a](1^) respectively. 

We will be interested in certain cohomology classes arising from ni^. By 

Lemma 2.1 ni^ ~ (g) as 0{W) x GL(£J)-modules. Furthermore, see for example 
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[9], p. 80, 

(6.1) /\\rf^) /\\W® E') ^ 0§^(W) ® ME'), 

as O(l^) X GL(£')-modules. Here the sum extends over all partitions ji ol r with 
at most dim W — m — 21 rows and at most dim E — i columns, and denotes the 

conjugate partition of fi. 

We will be mainly interested in the case r = ni. Then we can take n = iwn = 

, ^ (gin 

(£,£,..., £), so that = nwi = (n, n, . . . , n) and S^>{E') ^ lf\ E'j ~ C is the 
trivial (one-dimensional) SL(£J)-module. We obtain 



(6.2) Sb{W)®Sb'{E') ~ [/\"'(W^^')J~'"^ ' - [f\'\ri*^k 

as 0{W) X SL(£^)-modules. Here B = Bn^i is the filling of the Young diagram 
associated to described in section 3.2. 

To realize this isomorphism, we define a GL(l^) x GL(£') intertwining map 

(6.3) Tr,e : T'{W) ® T^'{W) ® T'{E') ^ /\' {W ® E') ® T^' {V) C C'^' 
given by 

Tr,t'{{wi ® ■ ■ ■ ® Wr) ® ® {v[ ® ■ ■ ■ ® v'^)) = {Wi ® v'-^) /\ ■ ■ ■ /\ {Wr ® v'^) ® W, 

where w G T^\W). We also write for r^^o- We immediately see 

Lemma 6.1. The map Tr/' is 0(W) x SL(i?) x Sr+tj x Sr-equivariant. Here the 
action of the symmetric group Sr+e (respectively Sr) is on the tensor factors involving 
W (respectively E'). 

For r = ni, the map realizes the isomorphism (6.2). Furthermore, 

Lemma 6.2. Let w e T"^+^'(l^) and V e T"^(S')- Then 

By slight abuse of notation, we view from now on Tne/' as a map of T"^'^^'{W) by 
setting 

Tn^,£'(w) := r„^,£'(w ® (m'i ® ■ ■ ■ ® «£)")• 
We let (W^I'^l) be the space of harmonic /c-tensors in V (W); i.e., the tensors 
which are annihilated by all the contractions Cij. We let £''{V) C T'^{V) be the 
orthogonal complement of the harmonic tensors. Thus £'^{V)is the sum of the images 
of the insertion maps Eij{gy) : T''~'^{V) T''{V), I < i < j < k with the metric gy 
of V. Similarly, we define S^W) C T^{W). Note §[A](Vr) C §[a](V'). However note, 
that if w e T^'{W) is a nonzero tensor in the orthogonal complement of T[^'l(iy) (i.e., 
spanned by tensors in the image of the inclusion with the metric for W), then w does 
not necessarily lie in the orthogonal complement in T'^'l (V) (since the metric of V is 
different) . 

Proposition 6.3. Let B again be the given filling of the Young diagram associated 
to £u!n and A be a filling for A. 



SL{E) 



nt 



SL(E) 
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(i) Let w e E)b\a{W). Then r„f/'(w) defines a cocycle in C"^'^'. More precisely, 
we obtain a map 

(ii) Let n < [ '^^™^ ] and let w e S[b|^](PF). Then the cohomology class 

[r„,,,, (w)]e//"^(np,§[^](y))^L(^) 
does not vanish. Thus we obtain an embedding 

(iii) Let w e Sp|a(W^) n£"^+^'(iy) 6e m t/ie orthogonal complement o/§[b|a](H^) 
inside §p|^(PF). T/ien 

[7r[A] o r„£,^'(w)] = 

in i?"^(np, S[yi](V^))- i^ere ttj^] is the natural projection from iJ*(np, §a(^)) 
to i7*(np, §[^](V)) induced by the orthogonal projection Sa(^) — ^ S[^](V). In 
particular, for w e E>B\AiW), we have 

i^A] orne,e'{w)] = [Tne,e{T^[B\A]{v/))]. 

Remcirk 6.4. The action of the torus A on the image of Tne,e' has weights a^^'*^, . . . , a^'^^ 

Proof. We give a rough sketch, (ii) follows from Kostant's theorem [17], which explic- 
itly decomposes i^'*(np, §[a](1^)) as a module of L. Namely, in degree ni representa- 
tions of highest weight of the form w{p + \) — p occur, where w is an element of the 
Weyl group for L of length ni. Here p is the half sum of the positive roots as usual. 
In our case, we have w{p + X) — p — Iwn ^\iox w — w\ - ■ -Wn with Wi — s^+j-i • • • , 
where is the associated reflections along the root a-^. 

(iii) of course also follows in principle from Kostant, but for that one would need to 
determine the complete list of representations which occur in i7"^(np, S[a](V^)), which 
is tedious. For later use, we record the primitives below. 

(i) follows from (ii) and (iii), but we briefly indicate a direct proof. First note that 
the action of o" G 5'^'/ on the coefficients T^ (K) commutes with the differentiation 
d: (i o (1 (g) (T (g) 1) = {\ ® a ® X) o d. Furthermore, in the first factor T"^(l^), 
Tni,ii factors through cp, the column anti-symmetrizer for Young tableau 5, that 
is, Tni^ai o [cb ® 1) — Tni/'. Combining this with Lemma 6.1 gives Tn£/i o {cb\a) — 
{l^CA)oTne,,' onT-'+''{W). 

Therefore it suffices to show that Tne/'{rB\A{^)) is closed. Indeed, we have 

d (Tn^/'(sp|A(w))) = d ((1 Ca) O T„^,£'(rp|A(w))) = (1 Ca) O d(T„^,£/(rB|A(w))). 

Furthermore, it suffices to establish closedness forn = 1. Indeed, if the Young diagram 
A arises from the partition {£[,£'2, . . . , ^^) of we write w = wi ■ • • (8)w„ e T'^^{W) 
with Wj G T^{W) and w = Wi (g) • • • w„ with G T^i{W). We then have a natural 
product decomposition 

(6.4) rne,e' (w (g) w) = r^/^ (wi (g) Wi) A • • • A r^,^; (w„ (g w„) , 
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for which d acts as a derivation. Now it is not hard to show that applying the 
differentiation d to the element T£_^'(w) with w e Sym^''"^ (W) gives rise to a map 



(6.5) Sym^+^'(W) ^ A (W E') ® {E[ (g) T^'-\W)) 

i=l 

I 

i=\ C 

Here E[ = <Cu[, and the sum extends over all Young diagrams C of size £ + 1, which 
have at least 2 rows (otherwise the dual diagram C would have at least £ + 1 rows, 
which is impossible as dimi?' = €). By the Littlewood- Richardson rule we now see 
that in the decomposition of '8>c{W)®T^ ~^{W) into irreducibles only Young diagrams 
with at least 2 rows can occur. Hence Sym^^^ {W) does not occur on the right hand 
side of (6.5), and the map vanishes identically. This proves (i). 

To prove (iii), we record the primitives. It suffices to show that for any w G 7""^^+^-2^ 
the form 7r[^] o Tnii'{sB\A{.Ei j{g^){w))) is exact. For this, it suffices to show that 
^^.1,1' B\AEi^j{g^){^))) is exact up to terms involving the inclusion of the metric gy 
into the coefficient system. The product decomposition (6.4) reduces the claim to 
the cases of n = 1 (in case the metric occurs in one factor for (6.4)) or n = 2 
(if occurs in two factors). We sketch the case n = 1 by giving the primitive 
exactly leaving n = 2 to the reader. We need a bit of notation. Let s < i. For an 
{i — l)-tensor w = Wi (g) Wg-i (8) Wg+i ®W(,& T^~^{W), we set 

Similarly, we define for a (£ — 2)-tcnsor w G T^~''^{W), the element t^^2^^\w) G 

Ae-2 
(W <S) E'), for 1 < si < S2 < ^ by omitting m'^^ and u'^^. For w = wi eg) 

• • • Wfe G T''{W), we let Sb{w) = ^ Y^aeSb '^t-H^) ® • • • ® Finally, we consider 

w = wi ® • ■ ■ ® u'£+£/_2, a (£ + £' — 2)-tensor. We write Pa for the set of multi-indices 
of size a inside {1,. + + 2}, consisting of the (ordered) sets / = {ii, . . . , ia} C 
{1, ...,£ + f + 2} of size a, we write w/ = (8) • • • Wi^. We also let / be the 
complement of / in {1, ...,£ + £' + 2}. Then 

^ll|rT£,^'(sB|A(-Bl,2(«/^)w)) - XI M^b{Wk)) ® SA{gv <^ "^k) 

KePe 

is exact with primitive 

+ 7 E E(-l)Va(.,-i(w,))®.^«®W7). 



e 

JePe-i s=i 



□ 
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6.2. The map ip. We now assume again that all objects are defined over M. 

We construct the map ip : ^ A*p of complexes mentioned in the introduction. 

We let U and U' be two complex representations of G and T : C/' — > C/ be G- 

intcrtwiner. We let C'(np, U) = { ^ n*p) U) be the complex computing the nilpo- 

tcnt cohomology ?i*(np, U), and we let C'i^g^{np, U) be the subspace of cocycles in 
C'{np,U). 

Lemma 6.5. Define a map 



by 



rf'^\J''^ (g) {J'^ ® u') = {J'^ A cj^'^) ® T{u'). 
Then rf'^ induces a map of relative Lie algebra complexes 

V : C (m, tp; C:,,Unp, U')) C'^' (p, Ip- U) 
and the induced map in cohomology factors through H* (m, Ip] H^{np, U')). 

Proof. This is essentially in [15], Lemma 2.6, see also [24], section 2, together with 
the standard spectral sequences argument in this context. Note that Harder actually 
considers instead of cocycles in C(np, U') the nilpotent cohomology group W{x\.p, U) 
reahzed as subspace in C(np, U') by harmonic forms as discussed in section 6.1. □ 

Definition 6.6. We define the map ip on Cy^''^ as follows. In fact, it is defined on 
the underlying tensor spaces without taking the group invariants. First we set ip to 
be zero if A; < nl. li k > ni we spht the two tensor factors 

We define tp on tensors which are decomposable relative to these two splittings. We 

let Ml = A ■ ■ ■ A On be the standard generator oi U — /\"'(C")* (with the twisted 
i^'-action). Let u{ ^ x ^ f ^ u ^ w he a single tensor component of an element in 
C^^' and assume that k > ni. Assume that x and w are decomposable, that is 

X = xi ® X2 e r'*^(c")* r'=-"^(C")* and w = «;i ® ^2 e r"^(Wc) ® t'^-^'^Wc). 

Then we define 
ip{u{ <S>x<^f<S>uj<S>w) 

Note here that by Lemma 3.2, we see that §b(C")* = s(B)*T"^(C")* ~ T^(f/)[0] and 
therefore u{^s{B)*{xi) lies in T^~^\U)[-^] and is zero if and only if s{B)*{xi) = 0. 

Proposition 6.7. ip is a map of complexes 

/^ji^fk Aj+£,r+n£,k—nt 

ip ■ ~^ ^p 



34 JENS FUNKE AND JOHN MILLSON 

Proof. In view of Lemma 6.5, it suffices to show that the map on C^''' to 

(6.6) r (m, tp;r'{np, T^-"^(Wc)) ® T^+'{U)[-E^] ® Wn,w) 
induced by 

(6.7) u{®X®f®UJ®Wi-^ {u{^ s{B)*{xi)) X2 ® f ® UJ ® rntiWi) ® W2 

gives a cocycle for the nilpotent rip-complex. Going through the proof of Theo- 
rem 6.3(i), we see that the composition of the tip-differential with (6.7) factors when 
viewed as a map on T'^(VFc) through representations Sc(Wc) with C having at least 
n + 1 rows. But now by Lemma 5.3 such representations do not occur in C^''^. □ 

The reader easily checks from the definition that tp satisfies the following properties. 

Lemma 6.8. (1) Lp is a [T{U) ® yVn,w ® /\pl^;-]^'^^^ -module homomorphism. 
That is, 

I W W \ W / W \ 

tp[iPj'yfi ■ ^j,r,k) — 'Pj'yfi • ^P[¥'j,r,k) 

for ^5r,, e cC.''-' ° and ^j;, e 

(2) ip{^f^r,k) ^^''^0 if k < n£. 

(3) Suppose <pYr,k ^ ^w''' ^^^^ k >n£ and <p^y^ii £ . Then 

(4) Let X e T"^(C")* and w e T^'^Wc). Then 

ip(lu ® a; (g) 1^ (g) Ipi;^ (g) w) = x{eB){ue Ic" (g 1^ (g lp;,, ® Tneiw) iT(yc))- 

Proposition 6.9. Let k = ni + i' as above. Let X be a dominant weight o/GL„(C), 
and we let A be a standard filling of the associated Young diagram -D(A) . We let B\A 
be the associated filling for the weight izUn + X, see section 3. 

(i) Then the preimage of A^p^''^'^^^''^ under bp lies in C^^'^^^ ;i.e., 

,-1 ( A3+t,r+ne.,A\ _ r^j,r,B\A 

ip y/ip j — 

Moreover, if ip{(p') = (p for ip' e c{^''^^+^' and ip e A/'^^'''^'^^'^\ then 

TTAif) = tp{TrB\A{(p'))- 

Here ttbia 'is the projection from (7^'"^+^ to C^'^^^, see (5.12), andnA is the 
one from A^''^^'^'''' to 

(ii) Let (p e j^o+(-,r+nt,[A] ^ closed form such that ip{(p') = ^ for some </?' e C^'^^^. 
Let 7r[B\A] be the projection from C^'^'"^ to C^'^^^'^K Then the cohomology 
class [ip] satisfies 

M = [Lp{niB\A]{<p'))]- 

Proof, (i) We ffist observe that tp is invariant under s{B) in the T"'^(H^)-factor and 
also s(i?*)-invariant in the T"^(C")*-factor, that is, 

ip = ip o (lu (g) ly„£(cn). (g (g Iw ® lp{y ® -"^(B) (g ^Tt'iw)) 

= ip o (![/ (g s{B*) (g lj.e'(^cny (g Iw ® ® 1t"«(w) ® lr<'(VK))- 
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Taking the ^'^/-invariance into account, we see tliat tp maps 
(6.8) 

T^{U) ® §b(C")* ® §a(C")* ® Wn,w ® /\'{p*w)c ® MWc) ® MWc) 
to ^i+2^.'^+"^.^. But now 

Lemma 6.10. 

(6.9) 



K'xKw 



K'xKv 



W 



Proof. In (6.9), we first observe §b(C")* (g) §a(C")* = §s|a(C")* as subspaces of 
T"^+^'(C"), see Corollary 3.4. But then by Schur-Weyl theory, see Lemma 5.3 or 
Remark 5.4, we can now replace Sb{Wc) ® Sa(Wc) with its subspace Sb\a{Wc) in 

(6.9), that is, the left hand side in (6.9) is equal to C^'^^^ 



□ 



Prom this we obtain Proposition 6.9(i). Proposition 6.9(ii) follows from Proposi- 
tion 6.3(iii) and Lemma 6.5. □ 



7. Special Schwartz forms 



Again, in this section, V will denote a real quadratic space of dimension m and 
signature (p, q). 

7.1. Construction of the special Schwartz forms. We recall the construction 
in [11] of the special Schwartz forms <fnq,£', <^nq,x, and ipnq,[x], which define cocycles 
in Cy^ , Cy^, and Cy'^^^ respectively. It will be more convenient to use the model 
Cy consisting of homomorphisms on ^^'(C") (and its subspaces St(A)(C")), see (5.7) 
and Remark 5.4. We will initially use the Schrodinger model <S(y") of the Weil 
representation. 

In [11], we construct for n < p a family of Schwartz forms (pnq/' on V"" such that 

'Pnq/' c Uy . t50 

K'xGxSf., 



K'xKxSf., 



(7.1) ipnq,e' e Hom(T^'(C"),T^(t/)®5(l^")®^"'^(D)®r^'(Vt)) 

Hom (r^'(c"), r«(c/) s{v") /\"'(py T^'{Vc)) 



These Schwartz forms are generalizations of the Schwartz forms considered by Kudla 
and Millson [18, 19, 20]. Under the isomorphism in (7.1), the standard Gaussian 



(^o(x) = 1 ® e-^*'-(^'^)-o e [rO(C/) (g) S{V'')f''^ corresponds to 



K'xG 
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Definition 7.1. Let n < p. The form (pnq,o with trivial coefficients is given by 
applying the operator 



^ n p+q ^ [ \ d \ 

i=l iJ.=p+l la=l ^ ^ 



to ifo: 



K'xK 



Here as before A{-) denotes left multiphcation and ui is the generator ol U — 

An 
(C")*. Furthermore, Theorem 3.1 of [18] imphes that ^Pnqfi is indeed X'-invariant. 

For T(Vc), we define for 1 < i < n another X-invariant differential operator V[ 
which acts on 

(7.2) S{V^)® f\{pl)®T{Vc) 
by 

(7.3) ^'^-\t{---Y.^)^^^^i-^)- 

Let / = (ii, . . . , i^/) e {1, . . . , nY' be a multi- index of length I' and write 

(7.4) £/ = £ii ® • • • ® Si^, 

for the corresponding standard basis element of ^^'(C"). Then for Si e T^'(C'^), we 
define an operator 

(7.5) %,{e,)^V[^o...oV',^, 
extend T^i linearly to T^'(C"). 

Definition 7.2. Define 

^nq,i' e Cr^'"' = Home (t'\C^),T%U) ® S{V^) ® f\\pl) ® T' {Vc)) 

by 

Vnq,e{w) = %'{w)ipnq,0 

for w e ^^'(C"). We put </?ng,^' = for I' < 0. Here the 5'^/-invariance of </7ng/' is 
shown in Proposition 5.2 in [11], while the X'-invariance is Theorem 5.6 in [11]. 

Using the projections T:t{\) and T^[t{X)\-i see (5.12) and (5.14), we can therefore make 
the following definitions. 

Definition 7.3. For any standard fiUing t{\) of -D(A), we define 

_ r^q.nq.X 

<Pnq,t{\) = T^t{\)Vnq,t' G , 

^ ^g,nq,[A] 
Vnq\t(\)\ = TT[t{\)]'^nq,e' t C/y 

We write (pnq,x and <Pnq,[X\, if we do not want to specify the standard filling. 



K'(^K(^Sf. 
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Proposition 7.4 (Theorem 5.7 [11]). The form ifnq/' is closed. That is, for w G 
T^'(C'*) and x e the differential form 



is closed. 

7.2. Explicit formulas. We give more exphcit formulas for (pnq,e' in the various 
models of the Weil representation. 

7.2.1. Schrddinger model. We introduce multi-indices ctj = {an, ■ ■ ■ , a^g) of length q 
(typically) with 1 < i <n and /3 = (/3i, • • • , /3^/) of length £' (typically) with values in 
{1, . . . ,p} (typically). Note that we suppressed their length from the notation. Wc 
also write a — {oiij) for the n x q matrix of indices. With / as above, we then define 

(7.6) OJaj^ — LUanp+1 A • • • A LUaigp+q 



Vnq,t'Mi^)^ [^"^ {d;T''{Vc)) 



~ T^aai O ■ • • O l~Laiqi-i 
'Ha — 'HaT_ O • • • O 
'Hpj = H/3iii O ■ ■ ■ O Hfi^ii^, 

= 6/3, (g) ■ ■ ■ O ep^, 



Let 1 < 7 < p and 1 < j <n. For /, a, and (5 fixed, let 

(7.7) = #{A;; akj = 7} + #{^; (/3fe, ik) = (7, j)}- 

This defines a. p x n matrix A.a,i3,i — ^a,i3,i;+ and Schwartz functions </?a, 
Definition 4.4. 



as m 



Lemma 7.5. The Schwartz form (pnq,e'{£i) is given by 






ul <8) {{Hai^ o • • • o Han ° 'Hp,i)(po) (8) {u;ai_ A • • • A a;aj (8) ep 



ai,...,an 




But now we easily see 



p 



n 



(7.9) 




7=1 j=l 



which gives the assertion. 



□ 
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7.2.2. Mixed model. We now describe the Schwartz form ipnq/' in the mixed model. 
We describe this in terms of the individual components (f^^ ^ j described in the 
Schrodinger model. Prom Lemma 4.7, Lemma 4.8, and Proposition 4.9 we see 

Lemma 7.6. 



U 



Note that (^X/ only depends on the indices aij,Pj such that cti,-,/?,- > i + 1, while 

a, 0,1 

(p^„ only depends on the indices CKij,(3j such that CKij,(3j < i. In particular, if all 

Ot,0,I 

o^ij, Pj > i + 1, then 

On the other hand, if one of the aij, (5j is less or equal to I, then 

/O 



xh/ I =0. 



7.2.3. Fock model. In the Pock model, the form (/p^^,^/ looks particularly simple. We 
have 

Lemma 7.7. 

1 /I / \ 

<^ng,r(gj) - 2ng/2+^' 1 2^ ) ^ «1 ^ai,l " " " ^^o^^n " (^m A • • • A W^j ® 6^. 

Here we use the notational conventions in (7.6) and in addition 

(7.10) ^Ojjj ~ ^Oijii ■ ■ ■ ^ajqjj ^/3,/ = ^I3iii ■ ■ ■ ^iSfiifi- 

7.3. The forms (po^k- We now define another class of special forms. We will only do 
this in the Pock model. 

Definition 7.8. We define ipo,k e Horn (T^(C"); TO(t/) O J^n,v ®T''{Vc)) by 
(7-11) <^o,fc(£/) = ^ j ^ ® ^i'^ ® 

Remcirk 7.9. The element (po,k is the image of the operator 7^ (see (7.5)) apphed to 
the Gaussian (pQ under the intertwiner from the Schrodinger to the Pock model. Also 
note that </7o,fe is not closed, hence they do not define cocycles. 

We also leave the proof of the following lemma to the reader. It follows (in large 
part) from Remark 7.9 and the corresponding properties of (finq/'- 
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Lemma 7.10. 
i.e., 

Prom Lemma 7.7, we immediately see 
Lemma 7.11. 

<Pnq,e' — <Pnq,0 ' <PO,e' 

and 

'Pom ■ VoM — VoM+k2i 
where the multiplication is the one in Cy. 

Remark 7.12. This kind of product decomposition for (pnq,e' and (/Po.fc in Lemma 7.11 
only holds in the Fock model. In the Schrodinger model this only makes sense in terms 
of the operators T> and %/ of Definition 7.1 and Definition 7.2 respectively. 

We apply the projection nt{\), see (5.12), to define (po,t{x)'- 

Definition 7.13. 

The following product formula will be important later. 

Proposition 7.14. Let A — t{X) be a filling of the Young diagram associated to X 
and let B = B^/ be the filling of the n x i rectangular Young diagram introduced in 
section 3. Then 

w w _ w 

Vo,B ■ 'Po,A — Vo,B\A- 

The proposition will follow from the next two lemmas. 
Lemma 7.15. Both ip^^ ■ and <^^|^ are elements of 

Proof Since Ss(C")* §a(C")* = Sb\a{C")* as subspaces of r"^+^'(C"), see Corol- 
lary 3.4, the claim follows in the same way as Lemma 6.10. □ 

Lemma 7.16. 

«B • s(A)eA) = ^^B\A{s{B)eB ® s{A)eA). 

Proof. This is a little calculation using Lemma 3.3 and Lemma 7.11. Indeed, we have 
Kb ■ {s{B)eB ® s{A)eA) = • {s{B)eB ® s{A)eA) 

= <pZe+e' (8) s{A)eA) = c{A, B)<p^^,^,, {s{B\A)eBiA) 

= c(A 5)(^SrB|A {s{B\A)eBiA) = <P^B\A (K^)^^ ® s{A)eA) . 

□ 

Now we can prove Proposition 7.14. By Lemma 7.15 we see that cp]^^ ■ cp]^^ 
and 'P^B\A C/(n)-equivariant homomorphisms from §b|a(C")* to T^{U) ® J^n,w ® 
^b\a{Wc). By Lemma 7.16 they agree on the highest weight vector (see Lemma 3.3), 
hence coincide. 
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8. Local Restriction 

We retain the notation from the previous sections. In this section, we will give for- 
mulas for the restrictions and rp of (pnq,t'- Finally, we will establish Theorem 1.1, 
the local restriction formula. 

Proposition 8.1. We have 



{r^vl,/'){^i) = 2^^^ Y.<® ® 



Eere ej — <S> ■ • ■ <S> e T^{C^), a' and (5' are the same indices as before with 

£ + l<«^,/?^<p. 

Loosely speaking, r-^ (v'ng,^') obtained from (pX,q,e "throwing away" all the indices 
less or equal to i. In particular, 



V _ Q 



if n > p — i. 



Proof. This follows from Lemma 7.5, the formula for (pnq,e in the Schrodinger model, 
and from Lemma 7.6. For the last statement, we observe that cUa' is in the ngr-exterior 
power of a (p — £)g-dimensional space. □ 

The local restriction looks particularly simple in the Fock model. We have 
Proposition 8.2. 

i^P^qA^l)) 

2Z '^\®^o\_---Za^- Zp^,I ® [Uja^ A • • • A 0;^^ j (g) 6^. 



2nq/2+e' y2TTi J 



Here a'j and (5' are as before in Proposition 8.1. 



Proof. This follows immediately either from Proposition 8.1 and applying the inter- 
twiner to the Fock model or also from Proposition 4.12 and Lemma 7.7. □ 

Proposition 8.3. For the restriction of (p^^^^,, we have 

rpvlq,v = (if/ ® Ic" ® rp ® a* ® Iv) (Pnq,e. 
Analogous statements hold for (fi^g^x ^'^'^ ^nq [A] • 

Proof. By Definition 5.5, the restriction rp : Cy —>■ Ap is given by lu <8) Ic" 8) r'p (8) 
{l* o a*) (8) ly. Then the theorem follows from Proposition 8.2 and Lemma 2.2, in 
particular (2.49): The components of involving a* already become annihilated 

under r'p, so that l* acts trivially on cr*r'pip]^g^f,. □ 
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We define 

1/1 

(8.1) (fip^ne = ^ ( ^ ) ® ^71,1 • • • ^7n,n ^ {l^j^^ ■ ■ ■ ^ I^jJ- 



7lv,7n 



Here 7^- = {jj^^i+i, . . . , ^j^) is a multi-index of length £ such that i + 1 < jji < p, 
and (7.10). Furthermore, we have set 

(8.2) i/^ = A • • • A t/^.^i e /\ (nt^). 

We have 
Lemma 8.4. 



Proof. First note that by Proposition 6.9 we have lp{'^^b) — ^pi^o'nt)- 

Pi,. . . be n indices of length £ with i + 1 < Pji < p. For the corresponding 

elements ep. e T^{W), we easily see 

(8.3) 

(%1 • • • Z0^n) ® Tniie^^ ® • • • ® e^) = {z^i ■ ■ ■ Zp^n) ® (i^^ A • ■ • A i/^) 

with i/^^ as in (8.2). With that, we conclude 
(8.4) 

^p(*) = (-ir^''^^^^2L(^)'^' E «l®(-m----^n)®(-^A---A.^) 

— — -l-j ^ VP,n£ 

by (8.1). □ 

Now Theorem 1.1 easily follows: 

Theorem 8.5. Let A he a standard filling of Young diagram with (! boxes and let 
S„ £ be the standard tableau associated to the n by £ rectangle as in section 3. Then 

rp{ip 

nq,v ri{q—t),nt+&)i 
rp{^lq,A) = ^pi.fn{q-t)MA)- 

Furthermore, for the form ip^^ with harmonic coefficients, we have 

[M'Png,lA])] = MVn{g-e),lB\A])]- 

Proof. We first note 

rp^nq,e' = (-1)^ ' ^ V)Jg-€),o • 'PPM ■ vie- 
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Here we view V^|^^_^) g ^ ^'"''^ ^^ '^ V^o^' ^ ^p°'^ in the natural fashion. The 
analogous statements hold for 92^^ yi and (p^^ Indeed, this follows immediately from 
Proposition 8.2 and 

(8.5) a*U^ = ^■■■^ ^oc'i,-,m-t A A ' ' ' A Va'^^u 

which follows from Lemma 2.2. The sign arises from 'sorting' a* [oJa\ A ■ ■ ■ A oJa'^ 

according to (8.5) into cJq,;'s (which lie in p^) and i^a'^^s (which lie in n^). From this 
and Lemma 8.4 we conclude 

By 5'^/-equivariance of ip we also obtain 

/ V \ t W W W \ w 

rp[^nq,A) = ^P\^n(q-i),Q ' 'Po,B ' ^0,a) = 'Pn{q-e),B\A 

since cp^^ ' = Kb\a (see Proposition 7.14) and ¥'5,_^),b|a = "Pniq-eio ■ V^b\a (see 
Lemma 7.11). 

The cohomology statement now follows from Proposition 6.9(ii). □ 

Corollary 8.6. We have [rp{<plg^[^{)] = for n > min (p, [f]) - £ (if £ > 2) and 
n > p — 1 or n > m — 2 — i{X) (if £ — 1). 

Proof. The Schur functor §[b|a](Wc) vanishes in this range. □ 
On the other hand, we have 

Corollary 8.7. Let P be a (real) parabolic subgroup as above such that the associated 
space W is positive definite. Assume 



i{X) <n< 

Then 



p-l-i(A) ifq^l. 



[rp{^Ux)] ^ 0. 



9. Global complexes, theta series, and the global restriction of 

In this section, we return to the global situation and assume that W, E etc. are 
Q- vector spaces. Furthermore, P is a standard Q-parabolic subgroup and P = Pq(^) 
for its real points etc. All the 'local' notions (over M) of the previous sections carry 
over naturally to this situation, and we make use of the already established notation. 



9.1. Global complexes and theta series. 



K 



Kp 
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9.1.1. Global complexes. We first define "global" versions of tlic "local" complexes 
C" of forms on X = T\D, A'p of forms on e'(P) = Tp\e{P). We set 

(9.1) C°°(r',j,A) := (r'\G";T^(t/) ® §a(C")*)^' 

for r' an (appropriate) arithmetic subgroup of Sp(n,Z). Note that we can identify 
C°°(r',j, A) in the usual way with the space of vector- valued C°°-functions on the 
Siegel upper half space of genus n, transforming like a Siegel modular form of type 
det^'/^®§A(C"). Furthermore, we denote by Mod{V',j,X) the space of holomorphic 
Siegel modular forms of this type. We let 

(9.2) C^'-'^ = C-(r',i, \)® [A'{D) ® ^^{Vc)f , 

~ c-(r',j, A)® [/\''(py ® §A(vt) ® ^^°^(r\G 

and 

(9.3) I^'-'" = C^ir'j, A)® [A'ie'iP) ® SxiVcf" 

~ C^{T',j, A)® [/\\n® PmTc ® Sa(^c) ® C^{Tp\P) 

We then define C^/'^^'^ and ij""''^^ as in the local case by harmonic projection onto 
S[A](Vc). The local map Lp induces a global intertwining map of complexes 

(9.4) lp : ^ 

by lifting functions on T]v\ SOo(W^r) to TmXM. This induces a map on cohomology 
lp : C°°(r, J, A) ® iJ"(''-^)(XH. §[^^„+A](l^c)) 

(9.5) ^ C^iT'j, A) H'^^^-'\Xm, H-\n, S[a](Vc))) 
^ C~(r',i, A) //"V(^), S[A](l^c)). 

We also introduce 

(9.6) C'/'^ = (r'\G';T^(f/) ® §a(C")*)'^'®^"(X,§a(Vc)), 

the complex associated to the differential forms on the compactification X with values 
in Sa(Vc), the local system associated to Sa(Vc). We then have a restriction map 

(9.7) rp-.C'y^ A'p 

induced by the inclusion e'{P) ^ X. 

9.1.2. Theta series. Using the Schrodinger model SiV^) of the Weil representation, 
we now introduce for ip e C^'^''^, its theta series 6'(v?) as follows. We fix /i e {L*Y 
in the dual lattice once and for all and assume that F C G(Z) stabilizes V^. We set 
C — Cv — Li^ -\-h. For g' e G' , we then define for z e D, the theta series 

(9.8) 9cy{g', z,cp)^Yl ""ig'M^, z). 



44 



JENS FUNKE AND JOHN MILLSON 



We easily see that the series is r-invariant as T stabilizes Cy Thus 6cy descends to a 
closed differential ng-form on the locally symmetric space X — r\D. More precisely, 
by the standard theta machinery, we have 

(9.9) ecAv) e Ci}^'' 

for some congruence subgroup F' C Sp(n, Z). Summarizing, the theta distribution 
9cy associated to C gives rise to a G" x G intertwining map of complexes 

(9.10) 9cy : C* &y. 

Remcirk 9.1. The main point of [11] is that for the Schwartz forms <^ng,[A] one has 
\QcAVn,xx])\ e Morf(r,j,A) ® iJ"^(X,§A(Vc)), 

and the Fourier coefficients are Poincare dual classes of special cycles with nontrivial 
local coefficients. 

For a similar theta intertwiner for Ap, we note that Ap involves the Weil repre- 
sentation for W — E^jE. Recall (see Proposition 4.2 and Definition 4.3) that we 

can extend the action of 0(W]r) on S(yV^ to P such that the Weil representation 
intertwining map becomes an MA^-intertwiner. In particular, N and M'p act 
trivially on S{W^). We let Lw be a linear combination of delta functions of (cosets 
of) lattices in VF", which is stabilized by Fp, that is by F^^. Recall that we defined 
V^J^f as the image of Fp when acting on E^ / E. It contains F H SOo(VFiR) as a finite 
subgroup of finite index. Applying the theta distribution asociated to Cw we obtain 
an intertwining map 

(9.11) Oc^ : A'p ^ A% 
Furthermore, Oc^^ commutes with tp: 

(9.12) Oc^oip^lpoec^. 
More general, we let 

(9.13) j^»^Cw,rw ^{^p^ A'p- Ocy^i^p) is F^^-invariant}. 

and obtain a map dc^ : A*^'^'^'^^ ^ as before. 

We will be interested in a particular Cw, which naturally arises from Cy as follows. 
We can write 

(9.14) CnE^^W {LI, + hE,k) © {Llr,k + hw,k) 



for certain lattices Le^u C E and Lw,k C. W respectively and vectors hE,k G {^%k)^ 
and hw,k £ {^wkY''- Considering C fl E^ modulo E, we then set 

(9.15) ^w^Y. det(i^E,fc)-" {Ll,^k + hw,k) , 

k 

viewed as a linear combination of delta functions on subsets of W. Here the deter- 
minant is taken with respect to the basis ui, . . . ,Ui of E. For this definition, it is 
crucial to view W = E-^/E as a subquotient of V (and not as a subspace). Namely, 
L'y^ f,+hw,k would only arise in WnL (when W viewed as subspace of V) if hE,k G L,^,k- 
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Remark 9.2. The definition oi Cw and its notation become more transparent (but a 
bit less explicit) if one changes to the adehc setting. Adehcally, C corresponds to the 
characteristic function xcy of the image of Cy inside V{Af), where A/ denotes the 
finite adeles. Then in this setting, Cw corresponds to the partial Fourier transform 
of XCv with respect to E{Af) when restricted to W. From this perspective, the 
assignment C — > Cw is the analogue at the finite places of the local restriction map 
rp at the infinite place. 

Theorem 9.3. The theta series 6cy{rp(pnq,e'), Ocv{^P}Pnq,\) , dcv{Vnq,[\]) extend to 
X, that is, 

Moreover, for a standard rational parabolic P, its restriction fp to the boundary 
component e'(P) is given by 

rp{Ocy{^nq,.)) = Oc^irp^nq,.)- 

In particular, rp{ifnq,») £ ^^'^w',rw_ 

Combining this Theorem 8.5, we obtain 
Corollary 9.4. 

fp{Ocy{^l^^e)) = '^p{Qcw^^n{q-t),nw))^ (<^n<j,A] ) ) = '^Pi.^Cw^^^q-e.U^r.+x))^ 

and 

[fp{Ocv{'^lq,[X]))] = Fp(%^(V^)!(g-£),[£^„+A]))]- 

Remark 9.5. The proof of Theorem 9.3 actually shows that (^CviVnqf) exponen- 
tially decreasing in the direction of e'(P) if n > p — £. In particular, Ocv{^nq,e') 
exponentially decreasing for n = p. 

Remark 9.6. More generally, the proof also shows that Ocy{^nq [X\) "almost" 
a special differential form in the sense of weighted cohomology, sec [14]. Namely, 
fp{(^Cv{vX.q [A])) iVp-invariant and while Ocv{vX.q [a]) ^"^^ neighborhood of e'(P) 
in X not the pull-up of the restriction via the geodesic retraction, the difference is 
rapidly decreasing. 

Proof of Theorem 9.3. It suffices to show this for ^X.q,i'- ■^o'^ 9 ^ G and g' G G', we 
let 

(9-16) Ol^^j{g',g)= ^ a;v(5')</^K„,,,,(^"'x) ® /a;„ ® ^e^. 

be the theta series associated to one fixed component of ^nq^^i- For the purposes of 
studying the restriction to e'(P), we can assume g' — 1 (since it intertwines with the 
restriction) and also g — a(t) e A (since g varies in a Siegel set and by Lemma 4.1). 
Then by Poisson summation we have 
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Lemma 9.7. Let a(t) G A. Then 



X |tl"<^f „ (t(C* + u'),tu')^^, (xvf) ® a(t) V*a;„ O a(t)e«. 

cx,0,I a,/3,J — 

Proof. This follows directly from the formulas given in Lemma 4.1. □ 



Write Aj = ai{a{t)) for the value of the rational root ctj for a(t). Then, in the 
non-split case (in particular, W 0) 

r+l-j 

(9.17) = n 

Lemma 9.8. Assume that at least one of the a^j and j3k is less or equal than i. Then 

rpOlsi = 0. 

Proof. By Lemma 9.7 and (9.17), we clearly see that each term in 0^p^j{a{t) is rapidly 
decreasing as Aj — > oo for a nontrivial root ctj for P unless both ^ — u — 0. But by 
Lemma 7.6, we have 

(9.18) ^1,,J0,0) = ^1^^^[Z)=0. 
in that case. 

In the split case of 0{p,p), everything goes through as above if ap = = 1. 
Otherwise 1^ = 0, and the puUback Ocyi'^^q^e.') ^'(-P) already vanishes. In this case 
we have tp — y^Xp/Xp-i and tp-i — ^ApAp_i. If exactly either ap or Up^i is trivial, 
then Ocv{Vnq,e') exponentially decreasing. If both are nontrivial, then considering 
Ap_iAp — tp ^ oo, we see that djCy{(fi!^g£,) is only bounded near e'{P). □ 

Now for the remainder of the proof of Theorem 9.3, assume that 

(9.19) «fc,,/5fc>£ + l. 

Again, each term in Lemma 9.7 is rapidly decreasing unless ^ = m = 0. So it suffices 
to consider 

(9.20) «(t)<^A„a, (y) " \^W\^^{y^w)®a{t)*a*uJa®a{t)e0. 
Now a{t)ej3 = ep by (9.19). We have 

(-1)^ 

(9.21) a*uj^ = A • • • A uJa^^_,m-l A i^a^^_,+^i A • • • A i/^.-^i, 
and A acts trivially on the a;,'s, while for the i/.'s we have 

(9.22) a(t)V,,= ^, 
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where I < i < £ and i+l<j<m — £. Here bji is the coordinate of W ® E for 
Cj (8) Ui and ti is the parameter in a{ti, . . . ,ti, . . . ,t£) & A. We obtain 

|t|a(t) a (Ja^ 2"^/^ |t|^aiip+i A • • • Aa;^^,^^^-^ A ^- A • • • A — 

A • • • 

A (^ai A • • • A UJa^_^ „m-e A A • • • A — 

— 2n£/2 "^"1 A ■ ■ ■ A ^m-^ A dba^_^^^ A ■ ■ ■ A d^aq a 
A ■ ■ ■ 

A A • • • A UJa^_^ ^m-l A C?&a,_,+i „M " " " A „1 

This shows for (9.20) we have 

(9.23) «(tVL,,, ( Y ) = ^i^^L,,. (^^) 

independent of t. This completes the proof of Theorem 9.3. □ 

9.2. Nonvanishing. We now prove Theorem 1.3. 

By the hypotheses wc can find a rational parabolic P_ such that dim E — £ — q, so 
W is positive definite and is a point. Then by Theorem 9.3, 

(9.24) [rpecAr, ^li,])] = M^cJ^, '/'oVn+A]] 

e Mod{r', m/2, A) (8) Zp {H\Xw, §[,^„+a](W^c))) 
~ Mod{V\ m/2, A) ® Tnq,e (§[^^„+a](W^c)) 
~ Mod{T', m/2. A) ® §[,^„+a](M^c). 

So in this case tp is an embedding. Hence the restriction to e'(P) vanishes if and 
only if the positive definite theta series 9^^{t, fo^^i^^+x]) vanishes. Furthermore, the 
restriction of the class [9cy{t, V^^[a])] cannot arise from an invariant form on D, since 
in that case one would need to obtain the trivial representation in the coefficients. 
To obtain the nonvanishing, we first observe 

Lemma 9.9. Given V^^^^^+aj above, then there exists a coset of a lattice Cw which 
we can take to be contained Cw such that 

Oc w 

Proof. We give a very simple argument which we learned from E. Freitag and R. 
Schulze-Pillot. We can assume V — with the standard inner product. First 
find a vector h G ]^(Z"*)" with Ni e Z such that 0- Then pick 

a lattice L = N^N^i^ such that || E.eL- V'oVn+A] (^) H' <"ll¥'&^„+A](^)ll- Such a 
G Z exists as V'o^to +a] ^ Schwartz function. Then the theta series associated 
to <^o^^^^+A] — L"' + h does not vanish. □ 
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From this data then, we now can find a CJy contained in Ly such that £V = C.w 
with ^£H^(r,¥'S;,^„+A]) ^ 0- Replace V with T n Stab L'. Then [fpQ Hyir^^l^^^^f^ ^ 0. 
This proves Theorem 1.3. 

One feature of our method to estabhsh non- vanishing is that we retain some control 
over the cover X', since this reduces to the very concrete question of non-vanishing 
of positive definite theta series. An easy example for this is the following. 

Example 9.10. Consider the integral quadratic form given by 

yiy'i + • • • + Vqv'q + 2x1 + ■■■ + 2x1 

with yuy'i,Xj e Z. So L = Z"^ with m^2q + k. Assume k > q. Note L* c |Z"*. 
We let r be the subgroup in Stab(L) which stabilizes L*/ L. Then 

Using our method this follows from the non-vanishing of the theta series 
V 1 1 Ti • • • r p4'^*(E^f)-r 
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